US  Army  Corps 
of  Engineers® 


Results-Oriented  Performance  Measures 
for  the  Civil  Works  Program 


REPORT  ON  PHASE  I 
(March  1995-February  1996) 


19961104  049 

April  1996  ”  ~  IWR  Report  96-R-16 

r*~wstRlBUTiON  StATEM^  A 

1  Appio'^®'^  public  release;  1  ^ 

1  "  l^tribulio^  UnEnuted  _i 


Results-Oriented  Performance  Measures 
for  the  Civil  Works  Program 


REPORT  ON  PHASE  I 
(March  1995-February  1996) 


Prepared  by 
Mark  Dunning,  Ph.D. 

U.S.  Army  Corps  of  Engineers 
Water  Resources  Support  Center 
Institute  for  Water  Resources 
Program  and  Analysis  Division 


IWR  Report  96-R-1( 


PREFACE 


This  report  was  prepared  to  document  the  first  phase  of  a  comprehensive  initiative  to 
develop  results-oriented  performance  measures  for  the  Civil  Works  Mission  of  the  United  States 
Army  Corps  of  Engineers.  The  initiative  was  undertaken  at  the  direction  of  Major  General 
Stanley  Genega,  Director  of  Civil  Works.  During  Phase  I,  Don  Cluff,  Chief  of  Programs 
Management  Division,  provided  executive  leadership  and  direction.  Dave  Kenyon,  Chief  of  the 
Programs  Development  Branch,  provided  general  supervision;  and  Tim  Hiltz,  Programs 
Development  Branch,  was  project  coordinator.  The  Program  Analysis  Division  of  the  Institute 
for  Water  Resources  (IWR)  provided  analytic  support  for  the  initiative.  The  IWR  project  team 
included  Mark  Dunning,  Chief;  Rick  Sinclair,  project  manager;  Karla  Allred;  John  Brill;  Sandra 
Remer;  Frank  Sharp;  and  Richard  Whittington.  Kyle  Schilling,  Director  of  IWR,  provided 
general  management  oversight  of  IWR  support.  In  addition,  Ted  Kanamine,  and  Donna  Ayres 
from  the  Engineer  Strategic  Studies  Center  provided  meeting  facilitation  support.  Craig  Holt, 
Oregon  Department  of  Transportation;  and  Ed  Joseph,  General  Accounting  Office  acted  as 
advisors  and  provided  facilitation  support.  Jim  Davidson,  Operations  Plan  of  Improvement 
project  manager,  and  Dave  Lichy,  Navigation  Data  Center,  provided  important  insights  and 
lessons  learned  from  developing  results  measures  in  the  operations  pilot  project.  Logistical 
support  to  conduct  and  document  the  proceedings  of  the  many  workshops  held  during  Phase  I 
was  provided  by  Planning  and  Management  Consultants,  Ltd.  Many  individuals  within  the 
Civil  Works  community  participated  and  contributed  their  time,  energy,  and  developmental 
thinking  to  the  Phase  I  effort.  They  are  identified  in  Appendices  1 , 2  and  6  of  this  volume. 
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SECTION  I.  INTRODUCTION 


Purpose 

In  FY  95  the  U.S.  Army  Corps  of  Engineers  Civil  Works  program  began  development  of 
program  performance  measures  in  response  to  the  Government  Performance  Results  Act  of  1993 
(GPRA).  This  report  describes  the  first  phase  in  the  development  of  these  measures.  It  presents  the 
goals  of  the  initiative,  the  results-oriented  performance  model  being  employed,  the  implementation 
process,  results  obtained  thus  far,  lessons  learned,  and  planned  next  steps  in  Phase  II.  The  report 
is  primarily  intended  to  serve  as  a  summary  reference  document  for  those  who  have  participated  in 
the  measures-development  process.  It  is  also  meant  to  provide  an  overview  of  the  goals  and 
methods  of  the  initiative  to  those  within  the  Corps  of  Engineers  who  have  not,  thus  far,  been 
involved  in  the  process.  Lastly,  the  report  is  meant  to  provide  information  and  lessons  learned  to 
others  involved  in  the  development  of  results-oriented  performance  measures. 

Goal 


The  Civil  Works  program  wants  to  ensure  that  the  products  and  services  it  provides  to  the 
nation  are  effective  in  producing  desired  program  outcomes,  meet  or  exceed  user  and  taxpayers 
requirements,  and  are  produced  efficiently.  The  goal  of  the  results-oriented  performance  measure 
initiative  is  to  develop  measures  that  gauge  the  performance  of  the  Civil  Works  program  in 
achieving  this  intended  result.  This  emphasis  stands  in  contrast  to  existing  performance 
measurement  which  monitors  internal  business  activities  (e.g.  how  much  money  has  been  spent,  how 
many  FTEs  used,  how  many  reports  processed,  etc.)  but  fails  to  provide  information  on  whether  the 
program  is  achieving  intended  results  and  fails  to  link  activities  measured  to  program  results. 

Results-Oriented  Performance  Measurement 

Results-oriented  performance  measurement  has  gained  attention  in  government  as  part  of 
the  debate  over  budget  deficit  reduction  and  government  program  accountability,  and  is  evident  in 
a  variety  of  current  administrative  and  legislative  efforts.  The  National  Performance  Review  and 
Reinventing  Government  have  stressed  the  importance  of  focusing  agency  efforts  on  program 
results,  customer  satisfaction  and  service  quality.  The  Chief  Financial  Officers  Act,  similarly, 
encourages  systematic  measurement  of  the  results  obtained  from  the  expenditure  of  public  funds. 
The  clearest  expression,  however,  of  the  results-oriented  management  philosophy  is  found  in  GPRA. 
GPRA  mandates  the  design  and  pursuit  of  "strategic  planning  and  performance  measures"  to  achieve 
better  results  from  public  agencies.  Senator  Roth,  one  of  the  co-sponsors  of  the  legislation  (along 
with  Senator  Glenn)  explained  GPRA's  intent: 

"For  the  first  time,  Congress  would  have  to  specify  what  the  measurable  outcomes  of  each  program 
should  be.  It  would  be  compelled  to  answer  the  questions:  'Just  what  is  it  we  expect  this  program  to 
accomplish?  How  will  we  know  if  it  is  performing  effectively  and  efficiently?'  Those  who  argue  that 
a  program's  mission  cannot  be  stated  in  measurable  terms  would  really  be  saying,  'We  can't  tell  you 
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how  to  determine  whether  this  program  works  well,  but  let's  fund  it  anyway  and  hope  for  the  best.’ 

That  response  should  not  be  acceptable,  and  would  not  be,  under  this  legislation." 

GPRA  mandates  three  activities: 

-Creation  of  strategic  plans  -  identifying  agency  missions  and  long  term  goals.  The  first 
strategic  plan  is  to  be  submitted  by  30  Sep  97. 

-Creation  of  annual  performance  plans  wherein  agencies  set  performance  targets  for 
achieving  their  long  range  goals.  The  first  performance  plan  is  due  in  Fall  1997  and  will 
set  performance  goals  for  the  FY  1 999  budget. 

-The  submission  to  the  Office  of  Management  and  Budget  (0MB)  and  Congress  of  a 
yearly  performance  report  presenting  the  actual  performance  compared  to  the  preceding 
year's  goals. 

GPRA's  provisions  set  the  stage  for  performance  budgeting.  That  is,  holding  agencies  accountable 
for  results,  and  linking  the  budget  process  to  performance.  GPRA  calls  for  performance  budgeting 
pilot  projects  in  1998  and  1999  to  evaluate  the  feasibility  of  incorporating  performance  budgeting 
into  GPRA.  0MB  has  already  begun  to  move  agencies  in  this  direction,  requiring  the  presentation 
of  performance  information  as  part  of  agency  budget  submittals. 

Operations  and  Maintenance  GPRA  Pilot  Project 

To  help  prepare  the  government  for  GPRA,  the  law  establishes  agency  pilot  projects  in  FY  1 994, 
1995  and  1996.  Pilot  projects  will  develop  and  test  procedures  to  comply  with  GPRA  requirements. 
The  Corps  Operations  and  Maintenance  (O&M)  Program  was  selected  as  a  GPRA  pilot  project.  The 
O&M  program  is  roughly  half  of  Civil  Works  in  terms  of  annual  budget  and  workforce.  This 
program  had  already  undertaken  a  comprehensive  assessment  of  its  business,  and  had  identified 
performance  measurement  as  a  program  improvement  goal  prior  to  the  passage  of  GPRA.  The 
O&M  program  identified  and  adopted  an  innovative  performance  measurement  approach  developed 
by  the  Oregon  Department  of  Transportation  (ODOT).  The  experience  and  insights  gained  from  the 
O&M  pilot  project  is  being  used  to  guide  the  comprehensive  development  effort. 
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SECTION  II.  CIVIL  WORKS  PERFORMANCE  MEASUREMENT 

MODEL 


The  model  employed  to  develop  performance  measures  is  an  adaptation  of  the  O&M 
performance  pilot  process,  and  is  based  on  the  following  relationships  among  key  concepts: 

-Results  are  the  intended  consequences  of  programs  -  i.e.  what  they  were  established  to 
accomplish.  Results  can  be  identified  on  the  basis  of  authorizing  language,  Administration  policy 
for  a  program,  and  by  focusing  on  the  needs  and  expectations  of  the  “customers”  of  the  products  and 
services  produced  by  the  program.  Customers  include  the  end  users  of  products  and  services,  but 
may  also  include  broader  categories  such  as  the  general  public  or  taxpayers. 

-Results  are  achieved  through  products  and  services  produced  by  the  program. 

-Results-oriented  performance  measures  help  identify  if  the  products  and  services  are 
effective  and  if  they  are  efficiently  produced. 

-Products  and  services  are  effective  if  they  are  consistent  with  authorizing  language, 
Administration  policy,  and  meet  customer  requirements  expressed  in  terms  of  performance  factors 
such  as  quality,  timeliness,  cost,  reliability,  etc. 

-Products  and  services  are  efficient  if  they  are  produced  at  the  least  cost  (measured  in  dollars, 
manpower,  or  time)  for  a  defined  level  of  quality  or  performance,  usually  expressed  on  a  cost  per 
unit  basis. 

Applying  the  model  essentially  consists  of  two  parts:  (1)  defining  the  performance  “targets”,  by 
specifying  the  results  that  CW  needs  to  achieve,  and  (2)  developing  the  measures  to  tell  how  well 
the  organization  is  doing  in  reaching  the  target.  The  first  step,  that  of  target  definition,  is  the 
province  of  executive  leadership.  Leaders  are  responsible  for  identifying  the  following  concepts: 

-CW  Mission:  Why  CW  exists.  What  CW  should  achieve. 

-Business  Programs:  How  CW  is  organized  to  carry  out  its  mission. 

-Business  Program  Purpose:  Intended  program  results  defined  by  (or  derived  from) 

authorizing  language,  and  administration  policies. 

-Key  Result  Areas  (KRAs):  Management  areas  of  emphasis  in  which  CW  must  excel  if  it 

is  to  achieve  and  sustain  results-oriented  performance. 

In  the  second  part,  the  process  of  developing  performance  measures,  program  specialists  develop 
measures  using  the  fi-amework  developed  by  leadership.  Two  types  of  measures  are  sought: 
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-Measures  of  Effectiveness:  Measures  which  help  determine  if  CW  is  producing  products 

and  services  which  are  fulfilling  authorized  purposes  and 
which  meet  customer  needs. 

-Measures  of  Efficiency:  Measures  which  help  determine  if  the  products  and  services 

are  being  provided  with  the  best  use  of  resources. 

The  CW  performance  measure  system  consists  of  several  organizational  levels  or  tiers.  Tier  I 
is  the  executive  leadership  level.  Tier  11  is  the  national  program  level.  Tier  Ill  is  the  district  level, 
and  Tier  fV  is  the  project  level.  Each  tier  below  Tier  I  is  expected  to  create  performance  measures 
which  help  focus  performance  at  that  level  in  ways  that  are  consistent  with  overall  Tier  I 
performance  guidance. 

Another  key  feature  of  the  CW  model  is  the  reliance  on  a  participative,  consensual  and  iterative 
process  to  develop  and  mature  the  performance  measurement  system.  In  this  approach  reliance  is 
placed  on  selecting  knowledgeable  persons  to  work  in  small  groups.  The  use  of  a  collaborative 
process  to  discover  and  develop  performance  measures  contributes  to  a  greater  sense  of  ownership 
of  the  measures,  and  a  commitment  to  making  the  measurement  system  succeed.  There  is  also  a 
recognition  that  the  measurement  system  will  evolve  over  time,  and  that  the  process  will  need  to  be 
iterated  several  times,  and  experience  in  using  it  accumulated,  before  a  results-oriented  performance 
system  is  fully  developed. 

A  Brief  Example 

The  model  can  be  illustrated  by  an  example  derived  from  the  Tier  I/Tier  11  process  performed 
in  Phase  1,  and  described  in  Section  II  below.  One  of  the  nine  business  programs  identified  by  Tier 
1  for  carrying  out  the  CW  mission  is  navigation.  The  navigation  program  exists  to  provide  safe, 
reliable,  and  efficient  waterborne  transportation  systems  for  the  movement  of  commerce,  national 
security  needs,  and  recreation.  To  achieve  this  intended  result,  the  CW  program  produces  two 
navigation  products  and  services  -  harbors  and  waterway  systems.  These  products  and  services 
serve  a  number  of  key  customer  groups:  shippers,  cargo  carriers,  military  transport/federal  agencies, 
recreational  boaters,  commercial  fishermen,  and  passenger  carriers. 

One  measure  of  the  effectiveness  of  navigation  products  and  services  in  achieving  intended  results 
is  : 


-Percent  of  waterway  availability,  defined  as  miles  of  waterway  closed  during  an  increment  of 
time  (weeks,  quarterly,  annually)  because  of  unscheduled  closures  divided  by  total  waterway  miles. 
This  measure  addresses  customer  needs  for  reliability  and  minimized  delays. 

An  example  of  an  efficiency  measure  is: 

-Ratio  of  time  actually  required  to  complete  a  waterway  project  to  the  initial  scheduled  time. 
This  measure  uses  time  rather  than  dollars  as  a  unit  of  cost,  and  addresses  customer  needs  for 
timeliness. 
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SECTION  III.  KEY  ACTIVITIES 


Key  activities  undertaken  in  Phase  I  are 
shown  in  Figure  1.  In  addition  to  the  time  Figure  1.  Chronology  ofPhasel  Activities 
period  encompassing  Phase  I  (March  - 
December  1995),  early  preparation  and 
background  work  which  led  up  to  the  initiation 
of  Phase  I  is  also  described. 

Preparation  and  Background  Work 
(October  1993  -  March  1995) 

The  impetus  to  develop  a  comprehensive 
CW  results  management  focus  came  from  a 
niunber  of  sources.  First,  an  Institute  for  Water 
Resources  (IWR)  research  workunit  to  assist  in 
the  development  of  O&M  performance 
measures  provided  a  forum  for  the  broader 
discussion  of  application  of  O&M  concepts  and 
approaches  to  the  rest  of  CW.  Second,  a  goal¬ 
setting  exercise  undertaken  by  the  Acting 
Assistant  Secretary  of  the  Army  for  Civil 
Works  in  the  summer  of  1994  provided  the 
opportunity  to  engage  CW  leadership  in 
discussions  about  implementing  a 
comprehensive  results-oriented  management 
process  as  part  of  CW  goals.  The  Director  of 
Civil  Works  established  a  CW  goal  to  begin  the  process  of  developing  results-oriented  performance 
measures  in  September  1994,  and  placed  the  responsibility  for  doing  so  with  Mr.  Don  Cluff,  Chief 
of  Programs  Management  Division  (CECW-B). 

IWR  was  retained  by  CECW-B  to  continue  to  provide  assistance  in  the  development  of  an 
approach  for  developing  results-oriented  measures.  Options  and  a  recommended  approach  for 
implementing  results-oriented  performance  measures  were  presented  to  the  Chief  of  Programs 
Management  Division  in  a  concept  paper  in  February  1995,  and  were  subsequently  approved  by  the 
Director  of  Civil  Works  in  March  1995.  The  approved  approach  would  draw  from  and  adapt  the 
basic  O&M  model,  would  accelerate  the  development  process,  would  be  less  resource  intensive;  and 
would  be  oriented  toward  developing  performance  measures  that  enable  managers  to  manage  their 
programs  more  effectively. 

Initial  Planning  and  Kick-Off  Meeting 

After  the  decision  to  proceed  with  development,  CECW-B  and  IWR  began  intensive  planning 
for  the  process.  A  series  of  one-on-one  interviews  was  conducted  with  executives  within  CW  and 
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in  the  Office  of  the  Assistant  Secretary  of  the 
Army  (Civil  Works)  (OASA(CW)),  Office  of 
Counsel,  Real  Estate,  and  Resource 
Management  to  explain  results  management 
concepts,  the  upcoming  process,  and  to  obtain 
executives’  input  into  the  overall  design  of  the 
process.  A  “rules  of  the  game”  meeting  was 
also  conducted  in  April  in  which  executives 
from  DoD,  OASA(CW),  and  General 
Accounting  Office  (GAO)  were  invited  to 
explain  what  constraints  or  other  factors  needed  to  be  included  in  the  process  in  order  to  comply 
with  GPRA  requirements.  In  addition,  a  meeting  was  held  with  0MB  for  the  same  purpose.  Using 
the  information  obtained  fiom  interviews  and  meetings,  a  kick-off  meeting  with  CW  executives  was 
held  in  May  to  announce  the  initiation  of  the  overall  process,  and  to  lay  out  the  schedule. 

Tier  I 

Tier  I  consists  of  executives  fi’om  within  CW,  as  well  as  fi’om  Office  of  Counsel,  Real  Estate, 
Resource  Management,  Research  and  Development,  and  OASA(CW).  Additionally,  selected  leaders 
from  Corps  divisions  (Major  Subordinate  Commands  or  MSC),  and  district  offices  were  chosen  to 
serve  on  Tier  I  based  on  recommendations  received  in  the  one-on-one  interviews.  The  roster  of  Tier 
I  members  is  shown  in  Appendix  1 . 

Tier  I  Leadership  Meeting  (June  1995) 

The  objective  of  this  meeting  was  to  define  the  framework  for  the  entire  performance 
measurement  effort.  Leaders  focused  their  efforts  on  defining  a  succinct  mission  statement, 
identifying  business  functions,  determining  Key  Result  Areas,  and  establishing  principles  to  guide 
the  development  process.  Tier  I  adopted  a  consensus  model,  wherein  the  group  could  not  move 
ahead  without  receiving  a  “thumbs  up”  affirmation  from  all  participants.  Participants  also  endorsed 
an  “80%”  rule,  i.e.  agreed  not  to  strive  for  perfection,  but  agreed  that  80  percent  was  good  enough. 
In  addition,  a  “parking  lot”  for  issues  which  could  be  set  aside  for  later  discussion  and  resolution 
was  established.  This  allowed  progress  to  continue  on  the  task  at  hand. 

Tier  I  Accomplishments 

CW  Mission  Statement.  Working  in  small  groups,  participants  developed  several  mission 
statements  for  the  CW  program.  Conunon  themes  among  the  five  groups  were  identified.  Revised 
mission  statements  were  created  by  the  teams  focusing  on  common  themes,  and  a  series  of  mission 
statements  was  created  and  revised  by  the  entire  Tier  I  group.  After  a  number  of  iterations  the  group 
agreed  upon  the  CW  mission  statement  shown  in  Table  1 .  This  mission  statement  represented  a 
consensus  among  Tier  I  as  serving  as  the  basis  for  orienting  CW  performance.  The  mission 
statement  was  subsequently  modified  by  the  Acting  Assistant  Secretary  for  Civil  Works  to  reflect 
higher  order  linkages  of  the  Civil  Works  mission  to  national  security  considerations  (Table  2). 


Table  1 .  Civil  Works  Mission  Statement  Developed  in 
Tier  I  Leadership  Meeting,  June  ‘95 _ 

“The  Army  Corps  of  Engineers  Civil  Works  mission  is 
to  contribute  to  the  national  welfare  by  providing 
quality  authorized  water  resources  and  emergency 
response  programs  through  partnerships.” 
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CW  Business  Programs.  Tier  I  was  faced 
with  several  choices  regarding  how  to  organize 
the  CW  “business”  programs  which  support 
mission  accomplishment.  The  approach  to 
structuring  business  programs  has  profound 
implications  for  follow  on  efforts  to  develop 
performance  measures.  Three  business  struct¬ 
ures  were  debated: 

-by  budget  funding  accounts  (General 
Investigations,  Construction  General,  O&M, 
etc.); 

-by  technical  function  (planning,  engineering,  construction,  O&M); 

-by  program  area  (navigation,  flood  control,  etc.). 

Each  option  had  pros  and  cons.  Tier  I  concluded  that  defining  CW  business  areas  on  the  basis  of 
programs  was  preferable  for  demonstrating  CW  performance  given  the  desired  focus  on  program 
results  and  not  processes.  This  was  a  strategic  and  critical  decision  that  signaled  a  departure  from 
conventional  practice.  After  discussion  the  group  identified  nine  business  programs,  and  developed 
purpose  statements  for  each.  These  business  programs  represent  Tier  I’s  judgment  of  what  needs 
to  be  performed  to  carry  out  the  CW  mission.  The  purpose  statements  represent  Tier  I’s  views  on 
the  intended  consequences,  i.e.,  the  results,  that  each  business  program  area  should  strive  to  achieve. 
These  are  shown  in  Table  3. 


Table  2.  Civil  Works  Mission  Statement  as  Modified 
by  Acting  Assistant  Secretary  of  the  Army  for  Civil 
Works 


“The  Civil  Works  mission  of  the  Army  Corps  of 
Engineers  increases  national  security  by  its 
contributions  to  the  strategic  goals  of  promoting 
prosperity,  enhancing  security  and  promoting 
democracy.  Contributions  to  each  of  these  goals  are 
made  by  providing  and  maintaining  quality  water 
resources  projects  through  partnerships.” 
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Table  3.  Civil  Works  Business  Programs  and  Purposes  Identified  in 
Tier  I  Leadership  Meeting,  June  '95 _ 

Navigation 

Provide  safe  and  efficient  channels  and  harbors  for  movement 
of  commerce,  recreation  and  associated  activities. 

Flood  and  Coastal  Storm  Damage  Reduction 

Save  lives  and  minimize  the  level  of  property  damage  incurred 
by  floods  and  storms  using  both  traditional  and  non-traditional  measures. 

Hydropower 

Provide  reliable  and  cost-effective  power  services  to  power 
marketing  agencies,  Federal  Energy  Regulatory  Commission  and  others. 

Regulatory 

Protect  the  nation’s  wetlands  and  navigable  waters  through  the 
Army  permitting  process. 

Environment 

Ensure  environmental  compliance;  manage  natural  resources; 
provide  for  mitigation;  provide  for  restoration. 

Emergency  Management 

Prepare  for  and  respond  to  natural  and  national  emergencies 
under  Corps,  FEMA,  and  other  agency  authorities. 

Support  for  Others 

Assist  other  governmental  entities  to  achieve  their  mission 
consistent  with  the  capabilities  of  the  Corps. 

Recreation 

Provide  quality  public  recreation  opportunities  and  experiences 
compatible  with  other  project  purposes. 

Water  Supply 

Provide  and  manage  storage  in  Corps  reservoirs  in  conjunction 
with  other  purposes  for  beneficial  uses. 
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Key  Result  Areas.  Using  the  same 
procedure  described  above,  Tier  I  developed  a 
number  of  initial  lists  of  Key  Result  Areas 
(KRA’s),  discussed  and  grouped  them,  and 
ultimately  developed  the  KRA’s  shown  in 
Table  4.  These  KRA’s  are  focal  points  for 
orienting  and  measuring  performance.  In  Tier 
I’s  judgment,  CW  will  achieve  and  sustain 
results-oriented  performance  by: 


Table  4.  Key  Result  Areas  and  Example  Attributes 
Selected  by  Tier  1  Leaders,  June  ‘95. _ 

1 .  Program  Results  and  Justification 
-Damages  prevented 
-Acres  protected 
-Wetlands  preserved 
-ROI  (Return  on  Investment) 

-B/C  (Benefit/Cost) 


-producing  products  and  services  that 
provide  a  public  benefit  associated  With 
program  purpose  (e.g.  wetlands  preserved), 
and/or  sound  public  investment  principles  (e.g. 
benefit/cost); 

-producing  CW  products  and  services  that 
satisfy  users  (customers)  by  providing  quality, 
by  being  produced  in  a  timely  fashion,  within 
budget,  and  responsive  to  customer  expect¬ 
ations; 


2.  Customer  Satisfaction  (External) 

-Product  quality 

-Customer  expectations/involvement 

-Responsiveness 

-Timeliness 

-Cost  (Within  budget) 

3.  Management  Effectiveness  and  Efficiency 
(Internal) 

-On  time 
-Within  cost 
-Overhead  rate 
-Cost/unit 

-TLM  (Total  labor  multiplier) 


-producing  CW  products  and  services  effic¬ 
iently  -  on  time,  within  cost,  at  an  appropriate 
cost  per  unit; 

-by  creating  and  maintaining  the  kind  of 
CW  organization  that  attracts  and  keeps 
technically  competent  workers,  and  provides  a 
satisfying  work  environment  for  employees. 

Guiding  Principles.  Tier  I  did  not  have 
time  to  address  the  issue  of  guiding  principles; 
however,  the  group  delegated  the  responsibility 
to  Don  Cluff  to  develop  these  based  on  those 
prepared  in  the  O&M  process.  The  principles 


4.  Technical  Competence 

-Maintain  skills,  etc. 

-Quality  workforce 
-Flexibility  for  future 
-Recognize 

5.  Work  Environment 

-Safety 

-  ACOE  (Army  Communities  of  Excellence) 
Training 

-Diversity  -  Morale 


shown  in  Table  5  are  those  used  in  the  O&M 

process,  and  were  subsequently  endorsed  by  Tier  I  as  CW  guiding  principles  as  well. 


Tier  11 


The  actual  task  of  developing  programmatic  performance  measures  for  each  of  the  CW  business 
programs  was  the  responsibility  of  Tier  II  business  program  specialists.  Tier  I  leaders  were  asked 
to  provide  nominations  of  individuals  who  were  expert  in  the  technical  functions  of  the  business 
program  areas,  who  had  a  broader  “programmatic”  perspective  as  opposed  to  an  individual  project- 
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centered  focus,  and  who  were  creative  and 
forward-thinking.  From  the  list  of  nominees 
obtained,  Tier  II  workgroups  in  each  business 
area  were  assembled.  In  order  to  maintain 
continuity  between  Tier  I  and  Tier  II,  a  Tier  I 
“Champion”  was  identified  for  each  business 
program  to  serve  in  the  Tier  II  business 
program  workgroup.  The  list  of  Tier  II 
participants  is  provided  in  Appendix  2. 

Tier  II  Performance  Measure 

Workshops  (July  1995) 

Tier  II  Workgroups  worked  within  the 
framework  established  by  Tier  I.  Workshops 
to  develop  results-oriented  program  per¬ 
formance  measures  were  held  at  the  Casey 
Building  over  a  three-day  period.  A  facilitated 
process  similar  to  that  employed  in  Tier  I  was 
used.  The  nine  business  program  workgroups 
met  together  in  plenary  session  at  the 
beginning,  at  the  one-third,  and  also  at  the  two- 
thirds  point  of  the  meeting  to  receive  instruction,  and  to  discuss  general  issues.  The  rest  ot  the  time 
was  spent  in  separate  business  program  team  workshops. 

The  workshop  process  used  a  structured  process  to  derive  results-oriented  performance 
measures.  Participants  first  identified  the  essential  products  that  a  program  produced.  They  then 
identified  the  customers  for  the  products.  Table  6  presents  the  products  and  customers  identified 
by  each  business  program  workgroup.  The  third  step  was  to  identify  customer  needs.  These  three 
steps  set  up  the  structure  to  derive  results-oriented  measures.  If  a  measure  fell  outside  or  did  not 
align  with  a  KRA  it  would  not  be  defined  as  being  a  program  performance  measure.  Definitions 
as  well  as  an  identification  of  potential  data  sources  for  the  measure  were  developed  for  each 
performance  measure  that  met  screening  criteria. 

The  Tier  II  workshop  produced  a  rich  discussion  and  reflection  about  Corps  CW  mission, 
business  products  and  customers.  A  number  of  key  issues  surfaced  in  all  nine  workgroups.  Primary 
among  these  was  identifying  who  the  customer  was  for  CW  business  program  products.  Agreement 
was  readily  achieved  on  customer  identification  at  the  project  or  service  delivery  level;  however, 
neither  clear  understanding  nor  agreement  was  arrived  at  as  to  who  constituted  customers  at  the 
program  level.  There  was  considerable  debate  about  whether  Congress,  as  representative  of  the 
broad  public  interest,  was  a  customer  whose  needs  and  expectations  needed  to  be  considered.  Tier 
II  participants  also  grappled  with  whether  local  sponsors  of  projects  should  be  considered  as 
customers  or  as  partners,  and  the  role  of  special  interest  groups  who  influence  the  character  and 
scope  of  products  and  programs.  The  lack  of  resolution  significantly  affected  Tier  IPs  ability  to 
identify  program  results  from  a  national  perspective,  rather  than  simply  as  an  additive  collection  of 


Table  5.  Guiding  Principles  Endorsed  by  Tier  I 
Leaders. _ 

Results  NOT  Activities 

TEAM-based  measures  NOT  individual 

TEAMS  will  develop  WORKGROUP  measures 

EFFICIENCY  &  EFFECTIVENESS  measures 

LINKAGE  of  Performance  Measures  to  National  KEY 
RESULT  AREAS 

CUSTOMER  linkage  to  Performance  Measures 
“Yardstick”  NOT  “The  Answer” 
Continuous  IMPROVEMENT  PROCESS 
ANNUAL  review  of  measures 


10 


project  products  and  services.  A  number  of 

other  issues  surfaced  and  were  discussed  during  Table  6.  Products  and  Customers  of  Civil  Works 

the  workshop.  These  are  listed  in  Appendix  3.  Business  Programs - 

Those  issues  which  were  not  resolved  by  the 
end  of  the  workshop  v/ere  referred  to  Tier  I  for 
resolution. 

Tier  I  Review  (August  1995) 

Tier  I  met  again  in  August  to  review  Tier  II 
comments  on  the  business  purpose  statements, 
and  to  provide  additional  guidance  to  Tier  II  to 
facilitate  achieving  closure  on  the  development 
and  refinement  of  program  performance  meas¬ 
ures. 


Revised  Purpose  Statements 

Appendix  4  shows  the  issues  raised  by  Tier 
II  about  business  purpose  statements  and  Tier  I 
revisions  to  the  statements.  Tier  I  embraced 
many  of  the  Tier  II  refinements  to  the  purpose 
statements;  however,  in  other  cases  it  further 
clarified  its  original  viewpoint. 


Customers 


As  previously  noted,  one  of  the  key  issues  at 
Tier  II  had  been  identifying  the  customer  for 
program  products  and  services.  Some  groups 
had  been  reluctant  to  develop  measures  imtil 
they  had  received  further  clarification;  others 
had  come  to  their  own  conclusions,  and  had 
created  measures  based  on  their  decisions. 

However,  there  was  a  general  perception  among 
Tier  II  participants  that  clarification  concerning 
the  definition  of  customers  was  needed  at  the 
program  level.  Tier  I  was  advised  by  the 
workshop  facilitators  (Craig  Holt,  ODOT  and 
Ed  Joseph,  GAO)  to  not  provide  specific 
guidance  to  Tier  II  about  who  customers  were 
for  the  business  program  areas.  Based  on  their 
experience  in  working  with  other  agencies,  they 
advised  that  Tier  II  workgroups  would  either  debate  the  correctness  of  Tier  Ts  pronouncements  or 
develop  program  performance  measures  in  a  mechanistic  way.  Instead,  Tier  I  developed  an 


Business 

Program 

Products 

Customers 

Navigation 

Harbors, 

Waterway  Systems 

Shippers,  Cargo 
Carriers,  Military 
Transport/Federal 
Agencies, 

Recreational 

Boaters, 

Commercial 

Fishermen. 

Flood  &  Storm 

Damage 

Reduction 

Flood  and  Coastal 
Storm  Damage 
Reduction 

American  public  as 
represented  by 
President  and 
Congress 

Hydropower 

Electrical  Energy 
Capacity 

Preferred 

customers  through 
Power  Marketing 
Agencies 

Regulatory 

Environmental 
Protection  and 
Development 

General  public 

Environment 

Compliance, 

Mitigation, 

Maintenance, 

Restoration 

Public 

Emergency 

Management 

Preparedness  and 
Response 

DoD,  general 
public,  other 

Federal  agencies 

Support  for 
Others 

HTRW 

Remediation, 

Constructed 

Facilities, 
Management  and 
Technical  Services 

Federal  agencies, 
OMB  &  Congress, 
international 
organizations/ 
governments, 

Indian  tribes, 
private  US  firms 

Recreation 

Recreation 

Facilities,  Natural 
Resources 

Visiting  public, 
future  generations 

Water  Supply 

Municipal, 
industrial  and 
agricultural  water 
supply 

Local 

governments, 
water  management 
districts,  other 
permitted  users, 
companies  and 
utilities 
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illustration  and  a  series  of  questions  to  help  Tier  11 
in  their  further  deliberations  about  questions 
(Figure  2).  The  figure  indicates  that  at  the  program 
level,  customers  are  those  who  provide  or  influence 
authority,  mission  and  resources.  This  non¬ 
directive  approach  was  taken  by  Tier  I  to  ensure  a 
deliberative  discovery  process  by  Tier  n  in 
subsequent  work  to  further  screen  and  refine  the 
programmatic  measures.  The  guidance  provided, 
however,  make  it  clear  that  Tier  I  was  interested  in 
program  performance  measures  that  provide 
information  useful  to  those  who  authorize  pro¬ 
grams  and  allocate  their  resources. 

Tier  II  Verification  Workshops 

The  initial  Tier  n  workshops  were  successful  in 
the  creation  of  program  performance  measures; 
however,  several  additional  steps  remained:  (1)  screening  against  customer  needs;  (2)  screening 
against  implementability  criteria  (expressed  as  whether  the  measures  were  understood,  and  whether 
they  provided  information  that  could  be  used  in  decision  making);  and  (3)  developing  performance 
goals  for  each  measure.  Individual  business  program  workgroups  were  scheduled  for  one  day 
workshops  to  complete  these  requirements.  Most  workshops  were  conducted  during  the  August  - 
September  timeframe.  However,  the  water  supply  and  emergency  management  workgroups  were 
not  conducted  until  November  and  December  respectively.  Tier  II  workgroups  developed  a  total 
of  84  results-oriented  program  performance  measures.  Appendix  5  provides  a  listing  of  verified 
performance  measures  for  each  business  program  area.  A  more  complete  specification  of  each 
measure  is  provided  in  each  workgroup’s  team  documentation  booklets  (available  from  Rick 
Sinclair,  CEWRC-IWR-A). 


(Planning  /  Strategy) 


(Are  we  doing  the  right  filings?) 


(Service  Delivery) 


Direct  Recipient 


-Gusi  omer— 

(How  well  arc  we  (doing)  delivery?) 


Figure  2.  Tier  I  Guidance  on  Defining  Customers 


Selection  of  Measures  for  FY  96  Implementation 

At  the  completion  of  Tier  11  activities  measures  were  in  various  states  of  readiness  and 
useability.  Many  required  data  that  were  either  not  collected  or  while  collected,  were  not  readily 
reported.  For  example,  the  hydropower  measure  of  unit  availability  was  manually  entered  in 
powerhouse  logs.  Reporting  such  data  will  require  additional  effort. 

During  the  process  of  sorting  and  identifying  measures  for  implementation,  it  became  apparent 
that  successful  development  and  use  of  program  performance  measures  was  dependent  upon  clearly 
defining  “proponents”,  i.e.  a  person  with  managerial  responsibilities  for  the  business  program. 
Business  programs  of  hydropower,  regulatory,  recreation,  emergency  management,  support  for 
others,  and  the  environment  areas  of  natural  resources  and  compliance  have  such  proponents.  The 
business  programs  of  navigation,  flood  and  storm  damage  prevention,  the  environmental  areas  of 
restoration  and  mitigation,  and  water  supply  do  not.  Navigation  and  flood  and  storm  damage 
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prevention,  moreover,  represent  the  two  biggest  Corps  business  programs,  accounting  for  over  66 
percent  of  the  CW  budget,  yet  they  lack  a  program  proponent  who  has  the  responsibility  for 
evaluating  the  health  of  the  program. 

Since  there  was  no  person  with  clear  program  responsibilities  to  continue  the  development  and 
implementation  of  performance  measures  for  these  business  program  areas  the  schedule  for  fielding 
performance  measures  bogged  down.  This  problem  was  solved  by  the  designation  of  performance 
measure  proponents  for  all  business  programs.  Appendix  6  identifies  program  proponents,  and  their 
points  of  contact  (working  level  proponents).  Performance  measure  proponents  have  the 
responsibility  for  the  selection  of  program  measures  for  use  in  FY  96,  and  for  the  further 
development  of  program  measures  within  a  business  program  area. 

Implementation  was  defined  for  FY  96  as  the  complete  specification  of  measures,  writing 
implementation  instructions  and  deploying  measures  for  data  collection,  development  of  baseline 
information  on  the  measures,  and  the  reporting  of  the  status  of  baseline  creation  and  baselines  at 
periodic  in  progress  reviews.  Determinations  of  a  measure’s  readiness  for  implementation  were 
made  primarily  on  the  basis  of  data  availability.  This  was  a  pragmatic  decision  reflecting  the  limited 
resources  available  to  begin  the  implementation  process.  The  advantage  of  this  pragmatic  selection 
process  was  to  gain  experience  in  implementing  measures.  The  disadvantage  of  the  selection 
process  was  the  resulting  bias  towards  management  effectiveness  and  efficiency  measures.  This  bias 
is  not  surprising.  The  Corps  is  strong  in  measuring  processes  and  activities,  and  many  of  the 
processes  and  activities  already  being  measured  relate  to  the  focus  of  management  effectiveness  and 
efficiency  KRA.  Results-oriented  measures  in  the  other  KRAs  will  require  the  most  development 
work  to  achieve  implementation.  It  will  take  time  to  put  data  collection  and  reporting  systems  in 
place  for  these  newly  conceived  measures. 

Based  on  the  above  considerations,  proponents  identified  a  “short  list”  of  fifteen  measures  for 
implementation  in  FY  96.  The  list  of  measures  is  shown  in  Table  7.  Appendix  7  provides  a  more 
complete  definition  and  specification  of  these  measures.  In  some  cases  the  measures  selected  for 
implementation  have  been  modified  from  the  original  Tier  II  specification  so  that  information  could 
be  obtained  without  overburdening  the  field.  For  example,  the  navigation  measure  of  waterway 
availability  was  defined  in  a  way  that  will  require  the  collection  of  additional  data.  However,  data 
are  currently  collected  for  the  portion  of  the  measure  focused  on  lock  availability.  In  this  case  it  is 
possible  to  adopt  interim  measures  for  FY  96  use  while  longer  term  data  collection  strategies  for  the 
more  comprehensive  measure  are  worked  out.  In  other  cases,  performance  measures  and  data 
already  developed  as  part  of  the  O&M  performance  pilot  provided  a  reasonable  match  with  a  CW 
measure.  For  example,  the  environmental  (natural  resources)  measure  “Percentage  of  projects  in 
compliance  with  authorized  mitigation”  is  substantially  the  same  as  the  O&M  measure  “Percent  of 
mitigation  lands  achieving  the  intent  in  the  authorizing  legislation  or  relevant  Corps  decision 
document.”  In  this  circumstance,  for  FY  96  implementation,  it  makes  more  sense  to  implement  the 
O&M  measure,  and  defer  the  issue  of  the  fiirther  development  of  the  CW  measure  to  the  next  year. 
Finally,  in  some  cases,  proponents  determined  that  Tier  II  measures  were  not  ready  for 
implementation  in  their  present  form.  In  some  of  these  instances  proponents  substituted  other 
O&M-derived  performance  measures  for  use  in  FY  96.  In  the  Water  Supply  business  area,  where 
there  were  no  O&M  measures  to  substitute,  no  FY  96  measures  were  fielded. 


13 


Table  7.  Civil  Works  Results-Oriented  Performance  Measures  Recommended  for  Implementation  in  FY  96 


Business  Area 

Measure 

Flood  &  Storm  Damage 

1 .  Total  annual  flood  damages  and  total  annual  damages  prevented 

2.  Return  on  incremental  program  investment 

Navigation 

3.  Industry  delay  costs  due  to  unscheduled  closures 

4.  Unscheduled  closures 

5.  Lock  chamber  days  available 

Hydropower 

6.  Availability 

7.  Percent  unplanned  outages 

Recreation 

8.  Dollar  value  of  volunteer  effort 

Environment 

9.  Percent  mitigation  lands  achieving  the  intent  in  the  authorizing 
legislation  or  relevant  Corps  decision  document 

10.  Percent  of  major  findings  corrected  vs.  number  identified 

1 1 .  Percent  of  significant  findings  corrected  vs.  number  identified 

SFO 

12.  Customer  satisfaction 

Regulatory 

13.  Percent  of  all  actions  completed  within  established  timeframes 

Emergency  Management 

14.  Annual  cost  of  readiness  in  dollars  per  year 

15.  Annual  national  assessment  of  emergency  management  preparedness 
direct  labor  compared  to  CW  program  direct  labor 

Water  Supply 

Measures  deferred  in  FY  96 

Proponents  will  continue  with  the  development  of  a  full  suite  of  program  performance  measures 
in  Phase  II  of  the  CW  initiative.  These  activities  will  entail  reconstituting  Tier  II  groups  to  refine 
measures,  to  develop  new  measures  as  needed,  and  to  address  the  data  availability  difficulties 
described  above  which  limited  the  ability  to  field  measures  in  FY  96. 
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SECTION  IV.  LESSONS  LEARNED 


Phase  I  has  been  a  complex  and  fast-paced  effort.  This  section  describes  a  niraiber  of  lessons 
learned  fiom  the  conduct  of  Phase  I.  These  insights  will  be  applied  to  Phase  H,  and  are  provided 
for  the  benefit  of  others  who  may  be  engaged  in  the  development  of  results-oriented  performance 
measures.  Lessons  learned  are  presented  under  two  categories  -  findings  about  results-oriented 
measures,  and  findings  about  the  process  used  to  develop  measures. 

Findings  About  Results-Oriented  Measures 

1.  Results  oriented  management  is  a  powerful  perspective  that  promises  to  have  profound  impact 
on  the  CW program. 

A  major  Phase  I  “result”  of  the  introduciion  of  the  results-oriented  management  perspective  has 
been  the  first  steps  to  realign  the  CW  budget  by  business  program  area.  Chapter  4  of  the  FY  97 
budget  memorandum  (September  1995)  describes  the  CW  program  using  the  business  program 
areas,  and  presents  budget  information  on  this  basis.  Figure  3  presents  the  FY  97  budget  described 
using  the  new  fi'amework. 

Other  changes  are  likely  to  come  about  with  the  introduction  of  the  business  program 
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Legend: 

Nav>  Navigation 

FCSDR-  Flood  &  Coastal 

Storm  Damage  Reduction 

Hydro  -  Hydropower 

Reg  -  Regulatory 

Env  -  Environment 

EM  -  Emergency  Management 

Rec  -  Recreation 

SFO  -  Support  for  Others 


Figure  3.  FY  97  Civil  Works  Budget  Displayed  by  Business  Program  Areas. 

Note:  SFO  and  EM  business  programs  receive  reimbursable  funding  from  other  agencies;  budget  amounts 
shown  are  under-representations  for  these  areas. 
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perspective.  For  example,  the  Corps  CW  Research  and  Development  Program  is  taking  steps  to 
organize  the  delivery  of  R&D  products  and  services  according  to  the  CW  business  program 
framework.  Currently,  CW  R&D  is  conducted  within  a  number  of  research  programs  which  have 
been  developed  to  address  particular  needs.  These  programs  are  being  grouped  according  to 
business  area,  and  research  work  efforts  re-oriented  to  examine  and  prioritize  how  research  activities 
can  contribute  to  improving  program  results.  This  illustrates  precisely  one  of  the  goals  of  GPRA  -- 
focus  on  results  and  not  simply  activities. 

The  business  program  orientation  may  have  other  consequences.  Currently  CW  is  organized  into 
functional  “stovepipes”  (planning,  engineering,  construction,  operations,  etc.)  where  responsibilities 
are  technically  focused.  In  recent  years  the  stovepipe  organizational  structure  has  been  criticized 
as  being  too  insular  and  self-contained.  The  “discovery”  that  the  responsibility  for  managing  the 
two  biggest  CW  program  areas  reside  across  functional  stovepipes  may  further  contribute  to 
discussions  on  alternate  ways  to  organize  to  best  deliver  products  in  CW  business  programs. 

2.  Implementing  performance  measures  is  difficult 

While  the  process  of  creating  performance  measures  is  difficult,  the  work  is  creative  and 
challenging.  The  process  of  implementing  the  measures,  however,  is  just  hard  work. 
Implementation  necessitates  fully  and  unambiguously  defining  the  measure,  determining  data  needs 
and  availability,  assessing  the  cost  of  data  collection,  and  weighing  costs  against  the  benefits  to  be 
obtained  in  the  use  of  the  measure.  Data  collection  and  reporting  mechanisms  must  also  be 
developed.  Where  data  are  not  available,  data  definition  and  reporting  standards  must  be  identified, 
and  ways  to  routinely  report  and  use  the  information  worked  out,  once  again  in  a  cost  effective  and 
efficient  manner.  Those  involved  in  developing  performance  measures  need  to  recognize  that  they 
have  only  just  begun  once  initial  measures  have  been  created. 

3.  Implementation  requires  follow-on  leadership,  resources  and  priority-setting. 

CW  followed  an  accelerated  schedule  to  develop  the  initial  set  of  results-oriented  measures. 
There  was  a  recognition  that  follow  on  work  would  be  needed  to  resolve  technical  issues  with 
measures.  The  expectation  was  that  individual  teams  would  take  it  upon  themselves  to  carry  through 
with  the  development  and  would  drive  their  measures  to  completion.  This  did  not  prove  to  be  the 
case.  Team  members  returned  to  their  own  jobs  and  to  other  priorities.  In  several  instances,  team 
leaders  of  performance  measure  development  teams  did  not  have  a  program  proponent  with 
management  responsibilities  to  pass  team  work  products  on  to  further  direct  implementation  effort. 
As  a  result,  the  further  development  of  measures  did  not  occur  where  program  proponency  was 
absent.  In  retrospect,  a  complete  implementation  phase  should  have  been  designed  in  which  teams 
were  given  the  responsibility,  resources  and  priority  to  complete  measures  development. 

Another  significant  related  finding  has  been  that  business  programs  that  have  easily  identifiable 
proponents  were  able  to  mobilize  and  move  measures  toward  completion  more  quickly  than  were 
cross-cutting  business  programs  that  did  not  have  an  easily  identifiable  proponent.  Naming 
performance  measures  proponents  for  all  business  programs,  identifying  the  completion  of  measures 
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as  a  priority,  and  providing  additional  resources  for  the  task  has  proved  to  be  necessary  for  bringing 
Phase  I  to  closure. 

Findings  About  the  Process  Used  to  Develop  Measures 

1.  Participation  and  consensus  building  are  time  consuming. 

The  positive  aspect  of  the  participative  approach  employed  to  develop  performance  measures 
is  that  it  builds  positive  energy  around  the  concept  of  performance  measurement  and  invests 
participants  with  ownership  in  the  measures,  and  a  greater  commitment  to  their  successful 
implementation.  The  downside  of  the  approach  is  that  it  is  time  consuming  and  requires  a  great  deal 
of  time  to  explore  and  process  individual  input.  In  effect,  one  individual  can  prevent  a  group 
decision  by  refusing  to  move  from  a  “thumbs  down”  position.  Skilled  facilitators  are  necessary  to 
assist  the  group  in  moving  through  such  issues,  and  to  keep  groups  focused  and  productive. 

2.  Participative  process  requires  patience  and  expertise. 

The  Phase  I  process  employed  a  “self  discovery”  model  wherein  group  participants  were 
expected  to  rely  on  their  own  background  knowledge  and  expertise  to  develop  measures. 
Participants  were  provided  with  only  process  instruction  and  process  assistance.  Measures 
developed  thus  far  are  promising,  but  in  keeping  with  the  constraints  of  the  general  process 
employed,  are  not  mature,  fully  developed  measures.  There  is  a  need  to  remind  participants  and 
leadership  that  the  iterative,  self  discovery  model  being  employed  works  best  when  groups  can  gain 
actual  experience  in  using  the  results  of  their  deliberations,  and  can  iterate  the  process  a  number  of 
times.  However,  the  process  may  also  need  to  provide  greater  subject  matter  analysis  and  guided 
facilitation  in  group  deliberations  to  better  focus  and  facilitate  group  efforts. 

3.  The  self  discovery  process  needs  peer  review  and  critique. 

In  Phase  I  no  intergroup  exchange  and  critique  of  measures  was  built  into  the  performance 
measures  development  process.  In  part  this  omission  was  a  function  of  time  and  cost  constraints; 
however,  in  part  it  reflected  a  reluctance  to  subject  one  team’s  measures  to  anothers’  scrutiny  to 
emphasize  workgroup  ownership  of  performance  measures.  In  retrospect,  encouraging  such 
intergroup  inspection  and  critique  might  have  yielded  better  measures  overall,  by  providing  the  self 
discovery  process  to  operate  faster  and  have  the  benefit  of  multiple  points  of  view.  Future  group 
processes  should  include  time  for  sharing  and  critique  of  measures  among  business  product 
workgroups. 

4.  The  use  of  the  “80%  rule”  was  vital  to  the  success  of  the  process,  but  was  hard  for  many 
engineers  and  scientists  to  accept. 

There  was  a  tendency  to  want  to  go  for  “perfection”  in  group  activities,  and  thereby  use  up 
additional  time,  in  trying  to  resolve  disagreements  over  minor  issues.  Skilled  facilitators,  and  the 
use  of  the  “parking  lot”  to  capture  important,  but  not  necessarily  central  issues,  was  essential  in 
keeping  participants  focused  and  productive.  Facilitators  who  have  familiarity  with  the  Corps  and 
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the  business  programs  further  enhance  workgroups’  staying  focused  and  on  track  than  facilitators 
not  familiar  with  the  technical  subject  matter  under  discussion. 

5.  It  is  important  to  pick  the  right  participants  for  the  workgroups. 

Workgroups  need  to  be  composed  of  participants  who: 

a.  Have  Positive  Energy.  The  participative  self  discoveiy  process  requires  substantial  energy 
and  dedication  on  the  part  of  participants.  The  process  was  aided  by  having  persons  who,  in  addition 
to  being  expert  in  CW  matters,  had  an  essentially  positive  outlook  on,  enthusiasm  for,  and 
commitment  to  CW.  Those  planning  to  use  a  participative,  self  discovery  process  should  emphasize 
to  participants  that  the  discussion  sessions  might  be  intense,  require  substantive  justification  of 
points  of  view,  and  necessitate  detailed  analysis.  It  also  helps  not  to  have  substitutions  for 
participants  inasmuch  as  a  “group  memory”  is  developed  regarding  the  evolution  of  a  performance 
measure  from  workshop  to  workshop.  Adding  new  members  slows  down  the  process. 

b.  Reflect  the  Full  Range  of  Business  Program  Functions.  The  workgroup  needs  participants 
with  expertise  in  both  product  creation  and  product  operation  phases  of  program  delivery. 

c.  Represent  Program  Proponents.  As  discovered  in  Phase  I,  program  proponents,  i.e. 
someone  with  management  responsibility  for  the  business  program  need  to  be  named  to  address  long 
term  implementation  requirements.  A  representative  of  the  program  proponent  should  be  on  the 
workgroup  to  better  ensure  the  smooth  transition  from  the  group’s  development  efforts  to  longer 
term  implementation  efforts. 

6.  Documenting  the  group  deliberative  process  was  difficult. 

Team  meetings  used  one  to  two  outside  persons  to  serve  as  “recorders”  to  take  notes  of  group 
deliberations  and  issues  considered  by  the  team  in  arriving  at  particular  conclusions.  Notes  were 
assembled  into  documents  to  provide  a  written  reference  for  further  group  work.  It  proved  to  be  a 
difficult  task  to  produce  written  records  of  group  deliberations  which  satisfied  all  team  members. 
Recorders  should  at  a  minimum  be  well  versed  in  the  substance  of  issues  being  considered  by  the 
group.  Subsequent  phases  of  the  project  will  explore  ways  to  improve  the  quality  of  written 
documentation  provided  to  teams,  to  include  turning  over  the  responsibility  for  documentation  to 
the  team  rather  than  outside  recorders. 
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SECTION  V.  NEXT  STEPS 


Phase  I  accomplishments  have  provided  a  good  start  on  what  will  be  an  iterative  process  to 
develop  a  fully  mature  system  for  results  management  of  the  CW  program.  The  next  step  in  this 
process  will  begin  with  Phase  II  efforts.  The  primary  goals  for  Phase  II  are  described  below. 

Reiterate  Tier  I 

The  Tier  I  leadership  team  will  be  reconvened  to  review  the  C  W  mission,  business  programs  and 
purpose  statements,  KRAs  and  guiding  principles  in  light  of  Phase  I  experiences. 


Reiterate  Tier  II 

Tier  11  will  be  formed  in  a  more  systematic  manner  to  ensure  that  participants  representing  both 
program  development  and  program  operations  aspects  of  program  performance  are  represented  in 
business  program  workgroups.  In  addition,  program  proponents  will  take  greater  responsibility  in 
the  selection  of  Tier  II  participants,  and  in  the  conduct  of  the  workshops  to  review  and  develop 
measures.  Tier  II  will  review  the  Phase  I  performance  measures,  and  will  be  asked  to  refine 
measures,  develop  new  measures  where  appropriate,  and  eliminate  measures  that  no  longer  seem 
useful. 

Establish  Performance  Goals  for  Tier  II  Measures 

Performance  goals  for  those  measures  for  which  baseline  information  can  be  assembled  will  be 
established  in  Phase  II. 

Institute  Tier  III 

A  process  to  develop  district  level  performance  measures  consistent  with  Tier  I  guidance,  and 
aligned  with  Tier  II  measures  will  be  undertaken  in  Phase  II. 
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ENVIRONMENT  PERFORMANCE  TEAM  MEMBERS 
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Joseph,  Ed  |  Process  Advisor  202/512-6492  _  202/512-6451 

*  Attended  Tier  II  July  17-19  workshop. 

**  Attended  both  Tier  II  July  17-19  workshop  and  August  23  workshop. 


FLOOD  AND  COASTAL  STORM  DAMAGE  REDUCTION  PERFORMANCE  TEAM  MEMBERS 


A-7 


A-8 


NAVIGATION  PERFORMANCE  TEAM  MEMBERS 
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REGULATORY  PERFORMANCE  TEAM  MEMBERS 


A-10 


WATER  SUPPLY  PERFORMANCE  TEAM  MEMBERS 
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APPENDIX  3 

PARKING  LOT  ISSUES  AND  RESPONSES 


1 .  This  appendix  provides  draft  answers  to  Parking  Lot  Issues  raised  by  participants  in  the  National 
Program  Performance  Measure  Team  Workshops,  17-19  July  1995. 

2.  Answers  are  identified  as  to  three  sources:  Tier  I,  Don  Cluff,  or  Staff.  The  attribution  “Tier  I 
“reflects  what  CECW-B  and  IWR  staff  (Hiltz,  Dunning,  and  Sinclair)  distilled  from  Tier  I 
discussions  at  the  15  August  workshop.  Tier  II  participants  should  keep  in  mind  that  Tier  I 
discussion  of  parking  lot  issues  was  done  as  they  re-evaluated  program  purposes  using  Tier  II 
feedback  and  items  1  .a.  and  1  .b  above. 

3.  The  following  illustrates  how  to  interpret  symbols  in  an  issue  and  answer: 

EXAMPLE: 

1.1.“  Can  we  call  our  offices  via  conference  call  to  get  consensus?  (FCSD)'’ 

“  1.1  =  Day  1,  Issue  No.  1  fi-om  the  17  -  19  July  Tier  II  National  Program  Performance 

Measures  Workshop 

(FCSD)  =  Tier  II  Program  Performance  Measure  Team  -  in  this  instance:  Flood  and 

Coastal  Storm  Damage  Reduction  Program 

Other  Tier  II  Program  Performance  Measure  Team  symbols  include: 

Nav  =  Navigation 

EM  =  Emergency  Management 

SFO  =  Support  for  Others 

Rec  =  Recreation 

HY  =  Hydropower 

WS  =  Water  Supply 

Env=Environment 

Reg=Regulatory 


PARKING  LOT  ISSUES,  Day  1 

1.1.  Can  we  call  our  offices  via  conference  call  to  get  consensus?  (FCSD) 

Cluff  Answer:  Short  answer  =  Yes.  Concern  however  was  resolved  as  workshop  process 
unfolded.  Process  assumes  empowerment,  sufficient  breadth  of  knowledge  within  work  group,  and 
an  iterative  process.  The  .80%  sufficiency  rule  and  iterative  process  allows  for  continuous 
improvement. 


A-15 


1.2.  There  are  two  missions  in  our  purpose  statement  (Flood  Damage  Reduction,  Coastal  Storm 
Damage  Reduction).  Should  these  be  separated  up  front?  (FCSD) 

Tier  I  Answer:  No.  Purposes  of  activities  were  believed  sufficiently  similar  as  to  allow 
combination.  If  performance  measurement  development  process  demonstrates  they  should  be 
separated;  then  it  can  be  done  in  next  iteration. 

1.3.  Should  emergency  flood  management  be  viewed  as  a  function  of  Flood  and  Coastal  Storm 
Damage  Reduction  Business  Program;  or  should  we  assume  that  falls  under  Emergency 
Management  Business  Program?  (FCSD) 

Tier  I  Answer:  It  is  appropriate  to  assume  emergency  flood  management  falls  under 
Emergency  Management  Business  Program  based  upon  the  way  the  Corps  of  Engineers  organizes 
its  resources  to  respond  to  the  business  purpose.  Tier  I  recognizes  the  cross-cutting  nature  of  some 
programs;  but,  Tier  I  decided  that  some  programs  are  special  emphasis  areas  within  the  Corps  CW 
mission  which  deserve  separate  identification.  The  merits  of  integration  or  recombination  can  be 
examined  as  Tier  I  and  II  gain  experience  in  results-oriented  program  performance  measurement. 

1 .4.  Is  this  for  real?,  i.e.  will  we  actually  use  these  to  manage  the  Corps?  (WS) 

Cluff  Answer:  Yes. 

(a)  There  are  no  guarantees  that  0MB  and  Congress  will  sustain  the  initiative  over 
time.  However,  GPRA  is  viewed  as  a  landmark  piece  of  legislation  and  a  sound  initiative.  Congress 
recognized  the  initiative  would  take  time  to  accomplish;  it  provided  for  a  six  year  period  to  progress 
through  pilot  projects  and  an  iterative  implementation  process  to  gain  experience  with  results- 
oriented  measures  before  full  implementation  in  FY  1999. 

(b)  0MB  is  proceeding  with  implementation  on  a  fast  track.  0MB  exercised  its 
prerogative  in  deciding  our  program  performance  measures  in  FY  96  ahead  of  the  results  of  Tier  II 
July  workshops.  0MB’ s  actions  pose  a  challenge  for  us.  We  either  seize  the  initiative,  develop 
measures  that  make  sense  to  the  Corps,  and  then  furnish  performance  information  to  influence  the 
deliberative  process  that  determines  budgets  and  appropriations.  Or,  alternatively,  we  can  let  0MB 
and  others  decide  for  us  what  our  program  performance  measures  will  be. 

(c)  Corps  C  W  leadership  fully  support  the  spirit  and  intent  of  the  act  and  are  moving 
ahead  without  depending  upon  guidance  or  direction  from  0MB  and  Congress.  The  Director  of 
Civil  Works  commented  in  the  Tier  I  and  Tier  II  workshops  that  once  you  understand  the  intent  of 
the  Government  Performance  and  Results  Act,  you  wonder  why  it  was  not  undertaken  years  ago. 
Measuring  results  based  on  business  purposes  is  a  far  stronger  way  of  presenting  Corps 
accomplishments  to  the  public  and  Congress  than  simply  reporting  on  processes  measures  such  as 
we  achieved  a  95  percent  expenditure  rate. 

(d)  Management  steps  to  date: 
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(1)  We  have  introduced  the  nine  program  structure  into  the  Budget  Memo 
for  FY  97  and  made  provisions  for  using  results-oriented  measures  based  on 
results  of  FY  95  workshops. 

(2)  The  FY  95  Chief  Financial  Officers  (CFO)  Act  Report  will  use  the  nine 
program  structure  to  begin  to  report  results  to  the  extent  we  can 
retrospectively  apply  measures  developed  in  FY  95  workshops.  We 
committed  to  doing  this  in  the  FY  94  CFO  Report.  The  Army  Audit  Agency 
will  critique  our  progress. 

(3)  We  are  beginning  integration  of  the  measures  into  the  Consolidated 
Command  Guidance  for  FY  96. 

(4)  The  Director  of  Civil  Works  directed  that  the  process  move  to  the  Tier 
III  (district)  level  in  FY  96  to  align  and  link  district  performance  measures 
to  the  national  programmatic  level  (mission  and  business  programs).  Our 
initial  ability  to  implement  and  use  results-oriented  program  performance 
measures  will  be  largely  influenced  by  the  progress  of  Tier  II  in  developing 
such  measures.  The  Director  of  Civil  Works  and  Tier  I  leadership  also 
recognize  that  it  will  require  an  iterative  process  to  gain  experience  with 
results-oriented  measurement  to  successfully  employ  them.  We  expect  that 
the  results-oriented  program  stmcture  and  performance  information  will  offer 
new  insight  into  the  ongoing  restructuring  initiatives  as  we  gain  experience 
with  results-oriented  performance  measurement. 

1.5.  The  Water  Supply  Business  workgroup  assumes  water  supply  for  emergency  management  is 
being  dealt  with  by  the  Emergency  Management  Business  workgroup.  Please  confirm  the 
assumption.  (WS) 

Tier  I  Answer:  The  assumption  is  appropriate. 

1 .6.  Program  Performance  Measures  should  not  be  tied  to  individual  performance  appraisals.  (Nav) 

Cluff  Answer:  Correct.  The  Corps  is  not  organized  such  that  an  individual  or  set  of 
individuals  is  responsible  for  the  results  of  a  program.  Many  of  the  programs  depend  upon  cross¬ 
cutting  functional  organizations  to  contribute  to  their  development  (planning,  design,  and 
construction)  and  then  operation.  Moreover,  each  program  is  an  aggregation  of  numerous  projects 
or  contributing  elements  that  exceed  the  span  of  control  of  an  individual  or  few  individuals  so  as  to 
be  able  to  relate  their  performance  appraisals  to  results-oriented  measures.  Results-oriented 
measures  will  heighten  the  importance  of  successful  matrix  management. 

However,  the  question  misses  a  central  goal  of  results-oriented  performance  measurement. 
Craig  Holt,  ODOT,  explained  how  results-oriented  performance  measurement  led  to  alignment  of 
activities  (organizational)  with  the  agency’s  mission  and  program  goals  when  implemented 
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vertically  through  out  the  organization.  Ed  Joseph,  GAO,  affirmed  this  was  a  common  finding  for 
both  private  and  public  sector  organizations  recognized  for  employing  best  management  practices. 

1 .7.  Should  there  be  another  program  area  ~  multiple  purpose  reservoir  management?  (A  complete 
issue  statement  follows)  (WS) 

Managing  Competing  Water  Uses  as  a  Business  Program 

An  essential  business  program  of  the  Corps  of  Engineers  is  the  balancing  of 
competing  uses  of  reservoir  storage.  For  multiple-purpose  reservoirs,  the  Corps 
mission  to  "contribute  to  the  national  welfare"  is  maximized  when  competition  for 
storage  is  managed  in  the  national  interest.  Individual  business  programs  maximize 
the  national  welfare  but  only  that  segment  of  the  national  welfare  which  they 
represent.  In  fact,  national  welfare  could  actually  be  diminished  with  aggressive 
pursuit  of  individual  business  programs. 

Seven  of  the  nine  business  programs  are  in  direct  competition.  Product  gains 
in  one  program  usually  require  product  losses  in  another  program.  This  is  because 
so  many  of  these  products  are  derived  from  a  common  and  limited  resource:  Corps 
management  of  reservoir  space. 

Performance  measures  established  for  individual  programs  are  inherently 
linked  to  each  other  by  shared  storage  space.  As  a  business  program,  balancing 
competing  purposes  can  have  its  own  set  of  performance  measures  serving  customers 
that  include,  but  are  not  limited  to,  those  of  other  business  programs.  In  the  end, 

Corps  performance  as  a  water  agency  is  measured  not  only  by  its  ability  to  manage 
individual  programs  but  by  its  ability  to  effectively  and  efficiently  manage 
competing  demands  for  stored  water. 

Tier  I  Answer:  The  multiple  purpose  issue  suggests  several  dimensions. 

(a) .  Should  seven  programs  be  subsumed  under  one  multiple  purpose  program  function? 
Or  should  a  tenth  multi-purpose  program  be  developed  where  the  seven  are  in  competition?  The 
issue  of  multiple  purpose  management  is  limited  in  its  application.  The  Corps  of  Engineers  is 
constrained  by  legal  authority  as  to  what  program  purposes  it  can  engage  in.  Multiple  purpose  as 
an  optimization  question  occurs  primarily  where  multiple  purpose  reservoirs  have  been  expressly 
authorized.  Multiple  purpose  reservoir  is  no  longer  a  significant  program  development  activity  for 
the  Corps.  The  implications  of  cost  sharing  introduced  in  the  1 986  Water  Resources  Development 
Act,  environmental  considerations,  and  diminishing  site  opportunities  with  economic  potential  for 
reservoir  development  make  the  future  of  multiple  purpose  reservoir  development  veiy  limited  with 
respect  to  new  project  starts. 

(b) .  Tier  I  did  not  support  making  it  a  focus  program  or  area  of  emphasis  in  the  first  round 
of  developing  results-oriented  program  performance  measurement.  Tier  I  is  open  to  having  the 
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question  pursued  further  in  subsequent  iterations  after  experience  is  gained  with  the  initial  program 
structure  and  performance  measurement. 

(c) .  Conversely,  for  those  projects  where  a  primary  purpose  is  authorized  (flood  damage 
reduction  or  navigation)  the  issue  of  potentially  competing  trade-offs  is  answered  by  the  limitation 
placed  on  the  other  program  purposes  authorities.  This  is  evident  in  the  qualification  "in  conjunction 
with"  for  water  supply  and  "compatible  with"  for  recreation  business  purpose  statements.  It  is 
present  also  in  hydropower  but  not  explicitly  identified  in  the  business  purpose  statement. 

(d) .  Given  multipurpose  water  management  does  not  have  a  significant  program  development 
future  (see  (a)  above);  does  the  operations  community  need  a  tenth  program,  separately  identified, 
for  developing  optimization  performance  measures  for  existing  multiple  purpose  reservoirs  operated 
by  the  Corps?  The  issue  of  optimization  typically  sorts  down  to  a  site  specific  or  region  specific  set 
of  solutions.  Resolution  typically  involves  developing  a  set  of  operation  plans  to  respond  to 
differing  hydrologic,  economic  and  political  considerations.  Corps  management  solutions  must 
reflect  state  water  law  as  it  relates  to  allocating  reservoir  storage  and  releases.  Thus,  reservoir 
managers  need  a  portfolio  of  operating  plans  to  make  the  trade-off  decisions  which  reflect  the  range 
of  physical,  economic,  and  political  considerations  unique  to  that  reservoir  or  system  of  reservoirs. 

(e) .  The  trade-off  or  optimization  challenges  of  multiple  purpose  reservoir  development  and 
operation  does  not  negate  the  need  to  have  a  better  articulation  of  outcomes  for  each  project  purpose. 
Tier  1  believes  the  more  the  multi-dimensional  nature  of  results-oriented  performance  measurement 
can  be  portrayed  for  each  program  purpose;  the  sharper  the  trade-off  analyses  can  be  displayed. 

(f) .  The  issue  statement,  coupled  with  issue  number  1 .8  (below)  suggests  a  strategic  planning 
question.  Should  the  Corps  of  Engineers  pursue  legislative  authority  to  take  on  the  role  of  multiple 
purpose  water  manager  for  the  Nation?  Corps  leadership  does  not  believe  this  is  an  appropriate  role 
to  pursue  given  the  primacy  of  state  water  law  in  most  aspects  of  water  resource  management;  and 
the  purposefully  constrained  authorities  granted  the  Corps  to  pursue  water  resources  development. 

1.8.  Should  we  collapse  many  of  the  business  programs  into  "provide  for  the  greatest  use  of  the 
nation's  water  resources"?  (WS) 

Tier  I  Answer:  No.  See  answer  7(f)  above. 

1.9.  Is  regulatory  a  business  process  or  part  of  our  environmental  and  navigation  missions? 

Tier  1  Answer:  Tier  I  recognizes  the  cross-cutting  aspects  of  regulatory  with  navigation  and 
environment.  However,  Tier  I  felt  the  activity  deserves  a  separate  program  focus  in  light  of  the 
political  importance  attached  to  it  by  both  the  Administration  and  Congress.  The  merits  of 
integration  or  consolidation  within  Navigation  and/or  Environment  can  be  re-evaluated  in 
subsequent  iterations  as  we  gain  experience  with  results-oriented  performance  measurement. 
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DAY  2 


2.1.  How  are  the  other  groups  viewing  their  customer?  Is  Congress  also  their  customer? 
(FCSD) 

Cluff  Answer:  The  question  of  whether  Congress  is  a  program  customer  has  been  the 
most  troubling  of  parking  lot  issues  to  resolve.  Tier  I  dealt  with  the  issue  in  their  August  15th 
workshop  and  offered  clarifying  guidance  to  Tier  II  for  their  subsequent  performance 
verification  workshops.  Tier  I  viewed  Congress  as  a  significant  but  not  the  only  customer  for 
many  programs.  The  advisors  to  Tier  I  cautioned  against  developing  lists  of  program  customers 
for  Tier  II.  The  focus  of  both  Tier  I  and  II  participants  should  be  directed  at  exploring  the 
question  of  which  key  customer  groups  influence  mission,  program  authorities,  and  resources 
and  then  focus  on  evaluating  their  information  needs  to  better  define  program  performance 
measures. 

2.2.  Can  a  tape  of  Craig  Holt's  opening  remarks  on  the  process  and  successes  with  the  process 
be  made  available  for  Tier  III,  IV,  etc?  (Rec) 

Cluff  Answer:  We  did  not  make  provisions  for  taping  Craig  Holt  facilitating  the  Civil 
Works  program  performance  workshops.  Holt  has  offered  to  make  available  tapes  of  him  in 
other  facilitated  programs.  We  will  review  his  other  tapes  for  possible  use  in  follow  on  Tier  III 
and  IV  workshops. 

2.3.  Groups  the  Corps  has  traditionally  called  customers  appear  to  be  categorized  as  co¬ 
developers  or  suppliers  (e.g.  businesses,  local  sponsors,  interest  groups)  in  this  process.  (Rec) 

Cluff  Answer:  See  the  answer  to  2. 1  above.  Some  of  the  confusion  and  disagreement 
appears  to  stem  from  interchanging  project  level  customers  with  program  level  customers. 
Businesses  and  local  sponsors  may  operate  at  the  project  level  to  be  co-developers  in  the  project 
development  phase  and  recipients  of  the  flow  of  services  during  the  operation  and  maintenance 
phase  of  a  project.  Interest  groups  representing  coalitions  of  business  interests,  project  sponsors, 
or  facility  users  may  operate  at  the  national  level  to  influence  mission,  authorities,  and  resources. 

2.4.  Will  there  be  provisions  for  researching  data  collection  and  management  of  performance 
measures?  (Rec) 

Cluff  Answer:  No  expressed  provision  was  included  in  GPRA  to  fund  the  development 
of  results-oriented  program  performance  measures.  There  are  no  separate  resources  identified 
for  data  collection  and  management.  This  does  pose  a  concern.  Resource  requirements  will 
have  to  be  evaluated  during  the  process  of  developing  data  implementation  guidance.  In  all 
likelihood  funding  will  come  fi'om  District  overhead. 

2.5.  There  appears  to  be  structural  inconsistency  and  overlap  in  some  of  the  program  purposes. 
(Rec) 
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Tier  I  Answer:  Tier  I  recognizes  there  may  be  overlap  in  some  activities  between  some 
programs.  Tier  I  selection  of  programs  was  based  on  authority  and  the  emphasis  the 
Administration  and  Congress  devote  to  activities  which  might  otherwise  be  subsumed  under 
other  programs,  such  as  regulatory,  environment  and  emergency  management.  Given  the 
iterative  process  built  into  the  GPRA  process.  Tier  I  and  Tier  II  can  evaluate  the  first  year 
experiences  with  results-oriented  measures  and  re-evaluate  program  boundaries  and  definitions 
when  we  reiterate  the  process  in  1996. 

2.6.  Do  we  want  to  be  in  a  national  leadership  role?  (EM) 

Tier  I  Answer:  The  program  leadership  role  lies  with  the  Federal  Emergency 
Management  Agency  (FEMA).  ^Me  operate  within  the  program  authorities  available  to  us  and 
under  the  leadership  of  FEMA. 

2.7.  Who  is  the  audience  for  the  purpose  statement?  (EM) 

Cluff  Answer:  It  is  intended  for  a  wide  range  of  audiences.  The  extent  of  the  range  is 
defined  by  who  and  what  organizations  have  an  interest  in  understanding  the  Corps  role  in 
emergency  management.  This  includes,  but  is  not  limited  to,  the  public  at  large.  Congress, 
FEMA,  Corps  employees,  and  Tier  II  workgroup.  The  purpose  statement  provides  the  needed 
context  within  which  program  performance  measures  are  to  be  developed. 

2.8.  Are  we  "management"  or  ""readiness"?  (EM) 

Tier  I  Answer:  Both.  Management  and  readiness  are  evident  in  the  two  thrusts  in  the 
purpose  statement.  Readiness  is  inherent  in  the  “prepare  for”  dimension  of  the  purpose 
statement.  Management  is  inherent  in  the  “respond  to”  dimension.  Editorial  note:  Tier  II  in  fact 
resolved  the  issue  themselves  by  identifying  two  products:  (1)  readiness  and  (2)  response. 

2.9.  Can  Tier  II  revise  the  purpose  statement?  Or  can  we  expect  response  to  the  pros/cons  from 
Tier  I?  (EM) 

Cluff  Answer:  Tier  II  program  performance  measure  teams  were  encouraged  to  provide 
critical  comment  and  recommendations  on  how  to  improve  program  purpose  statements. 
However,  the  prerogative  for  changing  the  purpose  statements  rests  with  the  Tier  I  leadership 
group. 

2.10.  We  would  like  to  see  results  of  Tier  II  prior  to  sending  them  up  to  Tier  I.  Is  this  possible? 
We  would  also  like  to  see  other  work  group  results.  (EM) 

Staff  Answer:  The  process  and  schedule  were  compressed  such  that  this  was  not 
possible.  A  separate  enclosure  in  this  information  package  reports  the  results  of  seven  of  nine 
program  performance  teams.  We  will  also  publish  a  close  out  report  for  this  first  year  iteration 
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which  will  provide  the  results  of  the  nine  program  performance  measure  teams,  lessons  learned, 
and  what’s  next. 

2.11.  What  is  the  role  of  Congress?  Stakeholders?  Interest  Groups?  (SFO) 

Staff  Answer:  See  answer  to  Day  2,  Issue  1  (2.1). 

2.12.  When  will  these  performance  measures  be  implemented?  (SFO) 

Cluff  Answer:  FY  96  for  those  measures  which  are  sufficiently  well  developed  and  do 
not  pose  data  collection  or  significant  resource  requirements. 

2.13.  In  O&M  we  developed  performance  measures  but  haven't  received  the  final  list  of 
measures  to  use.  Will  we  get  the  performance  measures  in  CW  at  the  field  level?  (HY) 

Cluff  Answer:  Absolutely.  [See  answer  to  2.10.] 

2.14.  What  to  do  with  performance  measures  that  fall  within  the  jurisdiction  of  another  business 
program  area?  (HY) 

Staff  Answer:  We  believe  this  will  turn  out  to  be  a  hypothetical  question.  We  believe 
the  screening  criteria  which  the  workgroup  will  apply  during  the  validation  and  verification 
process  will  preclude  any  such  possibility  to  clear  the  screening  process.  The  validation 
screening  proeess  focuses  on  assuring  performance  measures  align  with  the  program  purpose 
statement.  If  during  this  process,  a  measure  is  judged  appropriate  for  another  business  ftinction, 
it  will  be  a  simple  matter  to  refer  that  measure  to  the  other  program  workgroup. 

2.15.  "Customers"  should  define  performance  measures.  We  should  present  eaeh  identified 
customer  with  the  applicable  "purpose"  and  products  and  ask  them: 

-Do  you  think  you  are  a  customer? 

-Are  these  the  products  you  receive? 

-How  do  you  (customer)  measure  our  (Corps)  performance  in  providing  the  product? 
(Env) 

Cluff  Answer:  A  qualified  yes.  We  are  working  on  plans  to  eany  the  program 
performance  measures  to  the  Tier  III  and  IV  levels.  The  plans  include  provisions  for  drawing 
partners  and  stakeholders  into  the  dialogue  or  process  for  developing  project  and  service  delivery 
level  performance  measures.  Our  outside  advisors  caution  us  on  asking  partners  and 
stakeholders  to  lead  us  in  the  development  process.  This  will  not  work.  The  Corps  needs  to 
first  gain  understanding  and  experience  in  developing  performance  measures.  We  believe  we 
will  gain  valuable  insight  by  inviting  others  into  the  development  process;  but,  we  must  be  clear 
on  how  we  engage  them. 
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2.16.  Where  do  we  account  for  non-environmental  compliance  (OSHA,  wage/labor  standards, 
etc.)?  (Env) 

Staff  Answer:  For  the  present  we  view  these  as  falling  outside  the  focus  of  GPRA 
results-oriented  performance  measures.  They  are  internal  process  related  measures.  It  is 
possible  that  as  we  progress  into  Tier  III  and  IV  measures  that  such  performance  measurement 
concerns  will  emerge  through  the  KRA  -  Work  Environment. 

2.1 7.  Some  KRA’s  may  be  important  to  the  Corps  internally  (see  KRA#5:  Program  Results  and 
Justification)  and  to  external  customers  (see  KRA#1 :  Customer  Satisfaction).  Will  this  be  sorted 
out  at  Tiers  111  and  rV?  (FCSD) 

Staff  Answer:  These  two  KRA’s  are  important  at  all  levels;  measures  are  needed  at  the 
Tier  III  and  IV  level  as  well  as  Tier  II  level.  Program  results  and  justification  by  definition  focus 
at  Tier  II  level  program  performance  measurement.  However,  there  must  be  a  subset  of 
performance  measures  at  the  Tier  III  and  Tier  IV  levels  which  link  because  of  the  requirements 
of  aggregation  or  summation. 

2.18.  Would  you  give  us  feedback  on  our  KRA#5  (Program  Results  and  Justification)  analysis  - 
how  to  balance  results  appropriate  to  National/programmatic  level  with  political  realities  of 
micro-management?  (FCSD) 

Cluff  Answer:  We  do  not  expect  performance  measures  to  perform  the  balancing 
function  of  collective  versus  individual  concerns  of  Congress.  We  view  “political  realities”  as  a 
characterization  of  individual  Congressperson,  state  and  local  political  interests  regarding 
individual  projects.  That  reality  will  exist  regardless  of  whether  program  performance  is 
measured.  The  program  outcomes  perspective  provides  a  context  for  the  collective  body  of 
Congress,  the  Administration,  and  other  interests  to  assess  how  individual  project  performance 
relate  to  program  performance.  It  provides  the  saime  perspective  for  the  individual 
Congressperson,  state  and  local  interests  to  view  the  contribution  or  justification  of  an  individual 
project  on  a  national  level. 

2.19.  What  is  time  zero?  (FCSD) 

-When  we  deliver  the  project? 

-When  we  achieve  the  end  result  (our  purpose,  product)? 

-When  all  project  claims  are  settled? 

Staff  Answer:  First,  the  question  operates  at  the  Tier  III  and  Tier  IV  level  and  assumes 
project  construction  is  involved.  Second,  the  answer  depends  on  who  is  measuring  completion 
with  what  objective.  An  auditor  might  view  the  last  criterion  as  the  one  to  be  measured. 
However,  from  a  GPRA  perspective  it  is  the  customer’s  view  that  counts.  In  those  cases  where 
we  are  constructing  a  project;  “time  zero”  will  be  when  the  flow  of  project  benefits  becomes 
available.  This  may  involve  measuring  separable  project  elements  to  account  for  functional 
elements  becoming  available  before  project  completion. 
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2.20.  Where  do  we/should  we  measure  the  technical  quality  of  the  product/project?  It's  difficult 
to  see  this  at  the  District  or  project  level.  (FCSD) 

Staff  Answer:  The  five  Key  Result  Areas  do  not  identify  technical  quality  of  product  as  a 
separate  area  of  measurement.  The  Key  Result  Area  “Customer  Satisfaction”  does  anticipate 
that  Tier  II  will  develop  performance  measures  which  provide  for  obtaining  the  customer’s  views 
on  product  quality.  Such  a  measure  will  be  a  departure  for  many  in  the  Corps  where  emphasis  is 
placed  on  internal  measures;  i.e.,  standards  and  processes,  to  define  quality  or  assure  quality. 
Obtaining  customer  views  of  product  quality  provide  for  an  external  measure  of  quality.  We 
believe  the  concern  will  resolve  itself  as  we  progress  through  the  results-oriented  performance 
measure  development  process. 

2.21 .  KRA’s  #3  (Technical  Competence)  and  #4  (Work  Environment)  may  be  Tier  IV  measures. 
How  do  they  fit  at  Tier  II  programmatic  on  national  level?  How  will  HQ  track  them  if  they  are 
Tier  IV  measures?  (FCSD) 

Staff  Answer:  Our  approach  for  now  is  develop  the  plans  and  process  to  carry  the 
results-oriented  performance  measurement  down  through  Tiers  III  and  IV  and  gain  experience  in 
results-oriented  measures  developed  at  these  levels.  We  will  identify  those  measures  which 
allow  aggregation  to  reflect  a  program  perspective  and  assessment  as  well  as  individual  project 
or  local  conditions.  We  may  find  with  experience  that  these  two  KRA’s  do  not  link  to  programs 
as  defined  from  a  GPRA  perspective;  they  may  be  better  linked  under  a  Management 
Effectiveness  and  Efficiency  context. 
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APPENDIX  4 

REVISED  BUSINESS  PURPOSE  STATEMENTS 


NAVIGATION 

Original  Version  by  Tier  I 

The  purpose  of  navigation  is  to  provide  safe  and  efficient  channels  and  harbors 
for  movement  of  commerce,  recreation,  and  associated  activities. 

Proposed  Version  by  Tier  11 

The  purpose  of  navigation  is  to  provide  safe,  reliable,  and  efficient  key  waterway 
transportation  system  elements  for  movement  of  commerce,  national  security 
needs,  and  recreation. 


Revised  Version  by  Tier  I 

The  purpose  of  navigation  is  to  provide  safe,  reliable,  and  efficient  waterborne 
transportation  systems  (channels,  harbors,  and  waterways)  for  movement  of 
commerce,  national  security  needs,  and  recreation. 


FLOOD  AND  COASTAL  STORM  DAMAGE 

REDUCTION 

Original  Version  by  Tier  I 

The  purpose  of  flood  and  coastal  storm  damage  reduction  is  to  save  lives  and 
minimize  the  level  of  property  damage  incurred  by  floods  and  storms  using  both 
traditional  and  nontraditional  measures. 

Proposed  Version  by  Tier  II 

The  purpose  of  flood  and  coastal  storm  damage  reduction  is  to  save  lives  and 
reduce  the  level  of  property  damage  incurred  by  the  floods  and  storms  using  both 
traditional  and  nontraditional  methods. 


Revised  Version  by  Tier  I 

Tier  I  accepted  Tier  II  suggestions  and  proposed  version. 
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HYDROPOWER 


Original  Version  by  Tier  I 

The  purpose  of  hydropower  program  is  to  provide  reliable  and  cost-effective 
power  services  to  power  marketing  agencies  (PMAs),  the  Federal  Energy 
Regulatory  Commission  (FERC),  and  others. 

Proposed  Version  by  Tier  II 

The  purpose  of  hydropower  program  is  to  provide  reliable  and  efficient  electric 
power  to  the  nation  through  power  marketing  agencies  (PMAs). 

Revised  Version  by  Tier  I 

The  purpose  of  hydropower  program  is  to  provide  reliable,  efficient,  and  cost- 
effective  power  and  related  services  to  power  marketing  agencies  (PMAs)  and  the 
Federal  Energy  Regulatory  Commission. 


REGULATORY 

Original  Version  by  Tier  I 

The  purpose  of  the  regulatory  program  is  to  protect  the  nation's  wetlands  and 
navigable  waters  through  the  Army  permitting  process. 

Proposed  Version  by  Tier  II 

The  purpose  of  the  regulatory  program  is  to  balance  development  with  the 
protection  of  the  nation's  wetlands  and  navigable  waters  through  the  Army 
permitting  process. 


Revised  Version  by  Tier  I 

The  purpose  of  the  regulatory  program  is  to  balance  the  uses  and  protection  of  the 
nation's  wetlands  and  water  through  the  Army  permitting  process. 
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ENVIRONMENT 


Original  Version  by  Tier  I 
The  purpose  of  environment  is  to: 

•  Ensure  environmental  compliance  (ERGO) 

•  Manage  natural  resources 

•  Provide  for  mitigation 

•  Provide  for  restoration 

Proposed  Version  by  Tier  11 

The  purpose  of  the  environment  program  is  to  ensure  compliance  with  applicable 
laws,  to  mitigate  project  degradation,  and  to  maintain  or  improve  current 
environmental  conditions. 


Revised  Version  by  Tier  I 

The  purpose  of  the  environment  program  is  to  ensure  compliance  (ERGO,  etc.),  to 
mitigate  project  degradation,  and  to  maintain  or  improve  current  environmental 
conditions. 


EMERGENCY  MANAGEMENT 

Original  Version  by  Tier  I 

The  purpose  of  emergency  management  is  to  prepare  for  and  respond  to  natural 
and  national  emergencies  under  Corps,  FEMA,  and  other  agency  authorities. 

Proposed  Version  by  Tier  II 

The  purpose  of  emergency  management  is  to  take  a  leadership  role  in  preparing 
for  and  responding  to  natural  and  national  emergencies  supporting  the  Army  and 
the  nation  under  Corps,  FEMA,  and  other  agency  authorities  and  Executive 
Orders. 


Revised  Version  by  Tier  I 


The  purpose  of  emergency  management  is  to  prepare  for  and  respond  to  natural 
and  national  emergencies  under  Department  of  Defense  (DOD),  FEMA,  and  other 
agency  authorities. 
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SUPPORT  FOR  OTHERS 

Original  Version  by  Tier  I 

The  purpose  of  support  for  others  is  to  assist  other  governmental  entities  to 
achieve  their  mission  consistent  with  the  capabilities  of  the  Corps. 

Proposed  Version  by  Tier  II 

The  purpose  of  support  for  others  is  to  assist  other  entities  to  achieve  their 
mission  consistent  with  the  abilities  of  the  Corps. 

Revised  Version  by  Tier  I 

The  purpose  of  support  for  others  is  to  assist  governmental  and  other  entities  to 
achieve  their  mission  consistent  with  their  capabilities  and  expertise  of  the  Corps. 


RECREATION 

Original  Version  by  Tier  I 

The  purpose  of  recreation  is  to  provide  quality  public  recreation  opportunities  and 
experiences  compatible  with  other  project  purposes. 

Proposed  Version  by  Tier  II 

Tier  II  accepted  Tier  I  definition,  but  with  several  concerns. 

Revised  Version  by  Tier  I 

The  purpose  of  recreation  is  to  provide  public  recreation  opportunities  and 
experiences  compatible  with  other  project  purposes. 


A-28 


WATER  SUPPLY 


Original  Version  by  Tier  I 

The  purpose  of  water  supply  is  to  provide  and  manage  storage  in  Corps  reservoirs 
in  conjimction  with  other  purposes  for  beneficial  uses. 

Proposed  Version  by  Tier  II 

Tier  II  accepted  Tier  I  definition,  but  with  numerous  concerns  and  a  substantive 
“parking  lot”  issue. 


Revised  Version  by  Tier  I 

The  purpose  of  water  supply  is  to  provide  and  manage  storage  in  Corps  multiple 
purpose  reservoirs  in  conjunction  with  other  purposes  for  beneficial  needs. 


APPENDIX  5 

VERIFIED  PERFORMANCE  MEASURES 
BY  BUSINESS  PROGRAM  AREA 


EMERGENCY  MANAGEMENT 

1 .  Annual  aggregate  percentage  of  Corps  state  of  readiness 

2.  Annual  cost  of  readiness  in  dollars  per  year* 

3 .  Annual  national  assessment  of  emergency  management  preparedness  direct  labor 
compared  to  Civil  Works  program  direct  labor 

4.  Annual  aggregate  percentage  deviation  from  baseline  costs  and  schedule  for 
reconstruction  activities 

5.  Annual  aggregate  percentage  of  customer  satisfaction  for  emergency  management  events 

6.  Annual  aggregate  percentage  reflecting  the  percent  of  time  the  Corps  responds  within  an 
established  goal 

7.  Annual  aggregate  damages  prevented  in  dollars  versus  dollars  expended  for  flood  fight 
operations 

8.  Aggregate  of  annual  benefits  recouped  through  reconstruction  versus  the  total  cost  of 
reconstruction 

9.  Annual  national  assessment  of  emergency  management  response  direet  labor 
eompared  to  Civil  Works  program  direct  labor* 


ENVIRONMENT 

1 .  For  each  ERGO  chapter,  percent  of  projects  in  compliance 

2.  Cost  of  compliance  versus  cost  avoided 

3.  For  each  compliance  category  in  ERGO,  remaining  cost  to  achieve  100  percent 
compliance 

4.  Value  of  resources  mitigated  versus  cost  of  mitigation 

5.  Percentage  of  projects  in  compliance  with  authorized  mitigation 

6.  Percent  of  projects  in  compliance  with  part  I  (natural  resources  management)  of  the 
Operational  Management  Plan  (OMP) 

7.  Value  added  to  resources  managed  versus  cost  of  management 

8.  Value  of  resources  restored  versus  cost  of  restoration 

9.  Percent  mitigation  lands  achieving  the  intent  in  the  authorizing  legislation  or 
relevant  Corps  decision  document*  (Note  that  this  measure  was  drafted  during  the 
O&M  process  and  is  equivalent  to  measure  #6  above). 


*  Note:  Measures  identified  in  bold  have  been  recommended  for  data  collection  and 
benchmarking  in  FY  96  on  the  basis  of  procedures  described  in  pp.  12-13  of  the  Phase  I  report. 
See  Appendix  7  for  further  specification  of  these  measures. 
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FLOOD  AND  COASTAL  STORM  DAMAGE  REDUCTION 


1 .  Damage  prevented  divided  by  total  damage’*' 

2.  Cumulative  damage  reduction  divided  by  dollars  spent 

3 .  Time  from  start  of  study  to  initiate  project  construction 

4.  Customer  satisfaction  with  flood  and  storm  damage  reduction  product 

5 .  Percentage  of  projects  completed  on  schedule 

6.  Percentage  of  projects  on  /  under  /  over  budget 

7.  Total  average  annual  damages  prevented  by  our  projects* 

8.  Number  of  people  whose  probability  of  being  in  “harm's  way”  has  been  reduced 


HYDROPOWER 

1 .  Availability* 

2.  Percent  of  unplanned  outages  (forced  outages)* 

3 .  Percent  of  planned  outages 

4.  Derated  capacity  versus  installed  capacity 

5 .  Cost  per  megawatt  of  installed  capacity 

6.  FTE  per  megawatt  of  installed  capacity 

7.  Capacity  lost  due  to  nonpower  constraints 

8.  Revenue  lost  due  to  nonpower  constraints 

9.  Capacity  lost  due  to  outages 

1 0.  Revenue  lost  due  to  outages 

1 1 .  Capacity  lost  due  to  derated  or  worn  equipment 

1 2.  Revenue  lost  due  to  derated  or  worn  equipment 


NAVIGATION 

1 .  Percent  availability* 

2.  Harbors— estimated  total  project  cost  /  actual  total  project  cost 

3 .  Harbors — initial  total  project  time  /  actual  total  project  time 

4.  Ratio  of  project  benefits  to  Operations  and  Maintenance  (O&M)  costs 

5 .  Percent  transits  with  drafts  greater  or  equal  to  90  percent  of  authorized  depth 

6.  Percentage  of  the  world  fleet  that  can  use  U.S.  harbors 


*Note:  Measures  identified  in  bold  have  been  recommended  for  data  collection  and 
benchmarking  in  FY  96  on  the  basis  of  procedures  described  on  pp.  12-13  of  the  Phase  I  report. 
See  Appendix  7  for  further  specification  of  these  measures. 
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RECREATION 


1 .  Percent  cost  recovered 

2.  Ratio  of  non-Corps  costs/Corps  costs 

3 .  Number  of  partnerships 

4.  Percent  of  natural  resources  management  employees  perceiving  a  safe  environment 

5 .  Cost/unit  of  use  (camping) 

6.  Customer  satisfaction 

REGULATORY 

1 .  Acres  of  wetland  losses  proposed 

2 .  Acres  of  impact  permitted 

3 .  Percent  of  development  proposals  authorized 

4 .  Acres  that  would  be  impacted  without  the  program 

5 .  Regulatory  actions  to  provide  flexibility  and  cumulative  impact  analysis 

6.  Acres  compensated 

7.  Percentage  of  all  actions  completed  within  established  time  frames* 

8 .  Percentage  of  all  authorizations  completed  via  general  permits  and  abbreviated 
permitting  processes  during  the  reporting  period 


SUPPORT  FOR  OTHERS 

1 .  Customers  satisfied  with  timeliness* 

2.  Customers  satisfied  with  cost* 

3 .  Customers  satisfied  with  responsiveness* 

4.  Customers  satisfied  with  quality* 

5 .  Customers  satisfied  with  competence  of  workforce* 


WATER  SUPPLY 

1 .  Customer  service  rating  (AWWA) 

2.  Responsiveness  to  requests  for  water  supply 

3 .  Revenues  of  collected  water  supply  storage  /  revenues  owed 


*Note:  Measures  identified  in  bold  have  been  recommended  for  data  collection  and 
benchmarking  in  FY  96  on  the  basis  of  procedures  described  on  pp.  12-13  of  the  Phase  I  report. 
See  Appendix  7  for  further  specification  of  these  measures. 
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APPENDIX  6 

PERFORMANCE  MEASURE  PROPONENTS  AND 
POINTS  OF  CONTACT 


Business  Program 

CECW  Proponent 

Point  of  Contact 

NAVIGATION 

Dan  Burns 

CECW-O 

Barry  Holliday 
CECW-OD 

FLOOD  &  COASTAL 
STORM  DAMAGE 
PREVENTION 

Ed  Dickey 

CECW-P 

Bob  Daniel 
CECW-PD 

ENVIRONMENT 

Dan  Burns 

CECW-O 

Fred  Caver 

CECW-B 

Natural  Resources 

Darrell  Lewis 

CECW-ON 

Compliance 

Jim  Wolcott 
CECW-OA 
Restoration/ 
Mitigation 

Pete  Luisa 
CECW-BW 

SUPPORT  FOR  OTHERS 

Don  Kisicki 

CECW-I 

Don  Kisicki 
CECW-I 

HYDROPOWER 

Dan  Burns 

CECW-O 

Harold  Tohlen 
CECW-O 

EMERGENCY 

MANAGEMENT 

Dan  Burns 

CECW-O 

Ed  Hecker 
CECW-OE-D 

RECREATION 

Dan  Burns, 

CECW-O 

Darrell  Lewis 
CECW-ON 

REGULATORY 

Dan  Burns 

CECW-O 

Kirk  Stark 
CECW-OR 

WATER  SUPPLY  Steve  Stockton  Bill  Sorrentino 

CECW-E  CECW-E 


APPENDIX  7 

FACT  SHEETS  OF  RECOMMENDED  MEASURES 

EMERGENCY  MANAGEMENT 


PRODUCT:  Preparedness 


KRAs:  Customer  Satisfaction;  Management  Effectiveness  and  Efficiency 

MEASURE  #  2:  ANNUAL  COST  OF  READINESS  IN  DOLLARS  PER  YEAR 


Definition:  A  Corps-wide  assessment  of  expenditures  on  readiness. 


Demonstrates:  •  Training  costs 

•  Exercises 

•  Development  of  plans — communication  with  other  agencies 

•  Stockpiling  of  material 

•  Collection  of  information/data  sources 


Unit  of  Output:  Dollars  per  year 


Data  Source: 


Districts/Divisions/Labs/Headquarters 


U<-  ,  K 


PRODUCT;  Response 

KRAs:  Management  Effectiveness  and  Efficiency;  Program  Results  and  Justification 


MEASURE  #  9:  ANNUAL  NATIONAL  ASSESSMENT  OF  EMERGENCY 

MANAGEMENT  RESPONSE  DIRECT  LABOR  COMPARED  TO  CIVIL 
WORKS  PROGRAM  DIRECT  LABOR 


Definition:  Emervencv  management  direct  labor  ($)  for  response 

Civil  Works  direct  labor  ($) 

Demonstrates:  Annual  effort  for  response  compared  to  Civil  Works  efforts. 


Unit  of  Output:  Percent 

Data  Source:  To  be  determined 


A-37 


ENVIRONMENT 

PRODUCT:  Maintenance 


KRAs:  Management  Effectiveness  and  Efficiency,  Program  Results  and  Justification 

MEASURE  #6:  PERCENT  OF  PROJECTS  IN  COMPLIANCE  WITH  PART  I  (NATURAL 

RESOURCES  MANAGEMENT)  OF  THE  OPERATIONAL 
MANAGEMENT  PLAN  (OMP) 

(Clarification  provided  to  separate  natural  resources  fi-om  Part  II  of  the 
OMP,  which  addresses  Park  Management.) 


Definition:  The  number  of  projects  for  which  task  in  the  OMP,  including  those 

developed  in  cooperation  with  resource  agencies,  are  accomplished  as 
scheduled  divided  by  the  total  number  of  projects. 


Demonstrates:  When  the  management  of  environmental  resources  is  being  accomplished  in 

accordance  with  approved  schedules. 

Unit  of  Output:  Percentage  (yes/no) 

Data  Source:  Project  Maintenance  Office 

OMP 

Master  Plan 
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FLOOD  AND  COASTAL  STORM  DAMAGE  REDUCTION 

PRODUCT:  Compliance 

KRAs:  Management  Effectiveness  and  Efficiency 

MEASURE  :  PERCENT  SIGNIFICANT  FINDINGS  CORRECTED  VS  NUMBER 

IDENTIFIED.  (Significant  finding  requires  immediate  attention.  It  poses,  or 
has  a  high  likelihood  to  pose  a  direct  and  immediate  threat  to  human  health, 
safety,  the  environment,  or  the  mission.). 

Definition:  The  number  of  significant  findings  at  Corps  operated  projects  and  facilities 

not  including  outgrants,  that  werecorrected;  divided  by:  the  total  number  of 
significant  findings  that  were  identified  as  a  result  of  ERGO  assessments, 
regulator  inspections,  and  good  management  practices,  plus  any  uncorrected 
significant  finding  from  previous  year. 

Demonstrates:  Annual  compliance  with  environmental  laws  and  regulations  is  being 

accomplished. 

Unit  of  Output:  Percentage 


Data  Source:  Divisions/Districts/Labs 


PRODUCT:  Compliance 


KRAs:  Management  Effectiveness  and  Efficiency 

MEASURE  :  PERCENT  MAJOR  FINDINGS  CORRECTED  VS  NUMBER 

IDENTIFIED.  (Major  finding  requires  action  but  not  necessarily  immediate 
attention.  It  may  pose  a  threat  to  human  health,  safety,  or  the  environment.) 

Definition:  The  number  of  major  findings  at  Corps  operated  projects  and  facilities,  not 

including  outgrants,  that  were  corrected  divided  by:  the  total  number  of 
major  findings  that  were  identified  as  a  result  of  ERGO  assessments, 
relator  inspections,  and  good  management  practices,  plus  any  uncorrected 
major  findings  from  previous  year. 

Demonstrates:  Annual  compliance  with  environmental  laws  and  regulations  is  being 

accomplished. 


Unit  of  Output:  Percentage 

Data  Source:  Divisions/Districts/Labs 
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FLOOD  AND  COASTAL  STORM  DAMAGE  REDUCTION 

PRODUCT:  Flood  Reduction  and  Storm  Damage  Reduction 
KRAs:  Program  Results  and  Justification 

MEASURE  #1:  DAMAGE  PREVENTED  DIVIDED  BY  TOTAL  DAMAGE 

Definition:  Measures  cumulative  damage  prevented  by  Corps,  projects  divided  by  the 

cumulative  damage  occurring  in  areas  protected  by  Corps  projects. 

Demonstrates:  Effectiveness  of  program 

Unit  of  Output:  Percentage 

Data  Source:  Post-flood  reports,  FIA 

PRODUCT:  Flood  Reduction  and  Storm  Damage  Reduction 
KRAs: 

MEASURE  #7:  TOTAL  AVERAGE  ANNUAL  DAMAGES  PREVENTED  BY  OUR 

PROJECTS 

Definition:  Cumulative  average  annual  dollar  damages  that  all  completed  Corps  flood 

and  storm  damage  reduction  projects  (would)  have  mitigated  (if  the  design 
events  had  occurred). 

Demonstrates:  Damage  potential  absent  protection,  e.g.,  progress  against  time. 

Unit  of  Output:  $  cost  avoidance 

Data  Source:  Feasibility  reports,  DMs,  J-sheets. 
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HYDROPOWER 


, . . . .ST  •'  'Trr  •  ''-VT" ‘ 

PRODUCT:  Capacity 

KRAs:  Customer  Satisfaction  (External) 

Management  Effectiveness  and  Efficiency  (Internal) 

MEASURE  #1 :  AVAILABILITY 


Definition:  Availability  is  the  ratio  of  the  hours  that  the  unit,  power  plant,  or  system  was 

actually  on  line  or  capable  of  being  placed  on  line,  compared  to  the  total 
hours  in  the  period.  A  unit  is  considered  available  if  it  can  be  placed  in 
service  within  ten  minutes  of  request. 


Demonstrates: 

Unit  of  Output: 

Data  Source: 


Availability  is  a  measure  of  the  effectiveness  of  the  program. 
Percent  developed  by  available  hours  divided  by  period  hours. 
Power  plant  records 


t’"'' -  '  ‘  ■■  I./  f,  1., 

PRODUCT:  Capacity 

KRAs:  Customer  Satisfaction  (External) 

Management  Effectiveness  and  Efficiency  (Internal) 


MEASURE  #2: 

Definition: 

Demonstrates: 

Unit  of  Output: 

Data  Source: 


PERCENT  OF  UNPLANNED  OUTAGES  (FORCED  OUTAGES) 

The  ratio  of  the  total  hours  of  forced  outages  compared  to  the  total  hours  in 
the  period. 

Effectiveness  of  the  maintenance  plan 

Percent  developed  by  hours  of  forced  outage  divided  by  hours  in  the  period 
Power  plant  operating  logs 
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NAVIGATION 


PRODUCT:  Waterway  Systems 

KRAs:  Customer  Satisfaction;  Management  Effectiveness  and  Efficiency;  Program  Results  and  Justification 
MEASURE  #1:  PERCENT  AVAILABILITY 


DeHnition:  Days  Waterway  closed  X  20  miles  -  (waterway  mile  -  days  closed  for 

weather)  -  (waterway  mile  -days  closed  for  scheduled  repairs)  ^  365  days  X 
total  waterways  miles. 


Assumptions: 


A  closure  has  a  direct  effect  on  tows  within  1 0  miles  either  side  of  the 
closure.  Indirect  on  tows  further  away. 


Days  closed  for  weather:  Flood  flows  or  iced-in  closures  effect  the 
shippers  the  same  way  these  conditions  effect  the  Corps  (the  harbors 
and  docks  are  usually  effected  by  the  same  weather  conditions  and 
shippers  cannot  load  or  unload  during  these  times)  therefore  the 
shippers,  do  not  have  a  need  to  use  the  waterways. 


Days  closed  for  scheduled  repairs:  This  number  will  vary  significantly 
from  year  to  year  and  distort  the  results.  We  should  be  managing  to 
minimize  unscheduled  closure  on  the  waterways. 


Demonstrates:  Percent  of  time  waterway  system  is  usable  when  towboat  operators  want  to 

use  it. 

Manage  to  minimize  unscheduled  closure  time  of  locks  and  channels. 


Unit  of  Output:  Percent  waterways  system  available 

Data  Source:  LPMS  -  Navigation  Data  Center 

Channel  closure  data  is  reported  but  may  not  be  put  into  a  computer 
system  at  this  time. 
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NAVIGATION 


PRODUCT:  Waterway  Systems 

KRAs:  Customer  Satisfaction;  Management  Effectiveness  and  Efficiency;  Program  Results  and  Justification 

MEASURE  #5:  DELAY  COST  TO  TRAFFIC  ON  THE  NATION’S  WATERWAY 

SYSTEM 

Definition:  This  measure  will  be  calculated  by  summing  the  total  cost  of  delays  at  locks 

and  in  various  segments  of  the  waterway  system  due  to  lock  outages 
scheduled  maintenance  and  repair,  repair  due  to  accidents,  and  delays  caused 
by  congestion;  delays  to  tows  and  other  commercial  vessels  in  the  waterway 
segments  (not  channels  that  are  part  of  a  harbor  system)  that  require 
maintenance  dredging  in  order  for  vessels  to  pass  through  the  segment,  and 
delays  to  vessels  due  to  closed  channels  (usually  on  orders  from  the  U.S. 
Coast  Guard  Marine  Safety  Office)  for  reasons  of  high  water,  sunken 
equipment  in  the  channel,  substance  spills  or  other  accidents.  The  total  delay 
time  for  all  delayed  vessels  at  each  location  will  be  multiplied  by  the  average 
hourly  operating  and  capital  cost  of  an  average  size  tow  or  other  commercial 
vessel  and  summed  for  all  locations.  This  will  be  divided  by  the  sum  of  the 
ton-mileage  associated  with  that  river  segment  to  provide  a  measure  of  the 
volume  of  traffic. 


Demonstrates:  This  measure  highlights  the  impact  that  failure  to  be  able  to  maintain  the 

waterway  system  has  on  waterborne  transportation.  Some  of  the  delay  will 
be  the  responsibility  of  the  Corps  of  Engineers  and  will  reflect  normal 
scheduled  maintenance  operations  while  some  of  the  delay  will  be  driven  by 
natural  conditions  due  to  river  flows  and  by  accidents.  The  source  of  the 
delays  will  have  to  be  interpreted  by  analysis  of  the  individual  waterway 
segments  contributing  to  the  statistic. 


Unit  of  Output:  $/ton-mile 

Data  Source:  Lock  Performance  Monitoring  System  for  delays  at  locks.  Hydrographic 

Survey  or  River  Patrol  reports  for  channel  closures  or  reduction  below 
authorized  depths  and  widths,  U.S.  Coast  Guard  Marine  Safety  Officer  for 
closures  due  to  navigation  conditions,  e.g.,  high  water  during  floods,  bridge 
closures,  or  accidents. 
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NAVIGATION 


PRODUCT:  Waterway  System 

KRAs:  Management  Effectiveness  and  Efficiency 


MEASURE: 

Definition: 

Demonstrates: 

Unit  of  Output: 

Data  Source: 


LOCK  CHAMBER  DAYS  AVAILABLE 

Ratio  of  days  lock  chambers  are  operational  for  passage  of  vessels  to  total 
days  available. 

Availability 

Percent 

CMR  process 
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REGULATORY 


PRODUCT:  Environmental  Protection  and  Development 
KRAs:  Customer  Satisfaction;  Program  Results  and  Justification 
MEASURE  it  7: 


PERCENTAGE  OF  ALL  ACTIONS  COMPLETED  WITHIN 
ESTABLISHED  TIMEFRAMES 


Definition: 


Number  of  actions  completed  within  the  established  standard  timeframe 
divided  by  the  total  number  of  action  completed. 


Demonstrates: 


Program  efficiency/timeliness 


Unit  of  Output: 


Percent 


Data  Source: 


District  reporting 


PRODUCT:  HTRW  Remediation 

KRAs:  Management  Effectiveness  and  Efficiency 


MEASURE  #  1 :  CUSTOMERS  SATISFIED  WITH  TIMELINESS 

Definition:  In  the  perception  of  our  customers  are  we  delivering  products  in  a  timely 

fashion. 


Demonstrates: 

Unit  of  Output: 

Data  Source: 


The  degree  of  customer  satisfaction  with  timeliness. 

Numerical  rating  system 

Annual  survey  of  customer  representatives  (on  scene  coordinators  or 
Remedial  Project  Managers) 
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SUPPORT  FOR  OTHERS 


PRODUCT:  HTRW  Remediation 

KRAs;  Management  Effectiveness  and  Efficiency 


MEASURE  #  2:  CUSTOMERS  SATISFIED  WITH  COST 


Definition: 


In  the  perception  of  our  customers  are  we  delivering  products  at  reasonable 
cost. 


Demonstrates: 

Unit  of  Output: 

Data  Source: 


The  deg-ee  of  customer  satisfection  with  product  cost. 


Numerical  rating  system 


Annual  survey  of  customer  representatives  (on  scene  coordinators  or 
Remedial  Project  Managers) 


PRODUCT:  HTRW  Remediation 

KRAs:  Management  Effectiveness  and  Efficiency 

MEASURE  #  3:  CUSTOMERS  SATISFIED  WITH  RESPONSIVENESS 


Deflnition:  In  the  perception  of  our  customers,  are  we  responsive?  Do  we  seek  and 

incorporate  their  views;  treat  them  as  an  important  member  of  the  team; 
solicit,  listen  to  and  resolve  their  concerns;  display  flexibility  in  responding 
to  their  needs;  and  keep  them  informed. 

Demonstrates:  The  degree  of  customer  satisfaction  with  Corps  responsiveness. 


Unit  of  Output:  Numerical  rating  system 

Data  Source:  Annual  survey  of  customer  representatives  (on  scene  coordinators  or 

Remedial  Project  Managers) 
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SUPPORT  FOR  OTHERS 


PRODUCT:  HTRW  Remediation 
KRAs:  Technical  Competence 


MEASURE  U  4: 

Deflnition: 

Demonstrates: 

Unit  of  Output: 

Data  Source: 


CUSTOMERS  SATISFIED  WITH  QUALITY 

Are  customers  satisfied  with  the  quality  of  the  products  delivered. 

The  degree  of  customer  satisfaction  with  product  quality. 

Numerical  rating  system 

Annual  survey  of  customer  representatives  (on  scene  coordinators  or 
Remedial  Project  Managers) 
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